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FULL  SPECTRUM  TRAINING  AND  DEVELOPMENT:  SOLDIER  SKILLS  AND 
ATTRIBUTES 

EXECUTIVE  SUMMARY 


Research  Requirement: 

Counterinsurgency  (COIN)  is  essentially  a  human  endeavor  that  taxes  the  full  spectrum 
of  human  capabilities.  In  response  to  the  human  performance  challenges  of  COIN,  the  current 
crux  of  Full  Spectrum  Operations  (FSO),  the  U.S.  Army’s  training  and  development  efforts  focus 
increasingly  on  Soldier  attributes  such  as  problem-solving,  initiative,  and  accountability.  This 
report  reflects  that  focus  by  advocating  an  approach  to  Soldier  preparation  referred  to  here  as 
Full  Spectrum  Training  and  Development  (FSTD).  The  FSTD  approach  emphasizes  the  core 
skills  and  attributes  that  Soldiers  and  leaders  need  to  perform  adaptively  and  successfully  in 
FSO.  Methods  that  enable  an  FSTD  approach  include  peer-to-peer  (P2P)  training  and  outcomes- 
based  training  and  education  (OBT&E)  which  stress  preparing  Soldiers  and  leaders  to  think  and 
act  more  effectively  in  uncertain  and  complex  environments.  The  Army’s  mounting  adoption  of 
these  training  methods  creates  the  need  for  guidelines  and  educational  materials  for  institutional 
and  unit  settings. 

Procedure: 

The  Army  Reconnaissance  Course  (ARC)  was  selected  as  the  venue  for  this  research. 
Based  on  a  literature  review  and  observations  of  the  ARC  program  in  action,  P2P  training  and 
assessment  materials  were  developed  and  packaged  in  a  user-friendly  guide  to  facilitate  FSTD. 
Formative  evaluation  harnessed  a  multi-stage,  multi-source  approach.  Materials  were  vetted  by 
internal  subject  matter  experts,  behavioral  researchers,  OBT&E  advisors,  ARC  leaders,  and 
instructors  in  a  draft-review-revise  process.  Field  testing  followed  an  implcmcnt-asscss-re line 
process  that  involved  course  leaders  and  instructors.  Evaluations  by  users  followed  operational 
implementation  to  yield  feedback.  The  guide  was  revised  iteratively  based  on  the  feedback  to 
produce  an  adaptive  guide  for  the  ARC  cadre. 

Findings: 

The  Instructor’s  P2P  Learning  Guide  for  the  ARC,  including  job  aids  and  assessment 
tools,  was  useful  to  ARC  instructors  for  instilling  skills  and  attributes  in  reconnaissance  leaders. 
Feedback  from  instructors  was  generally  positive  for  all  areas  of  the  guide.  Many  respondents 
remarked  that  the  guide  is  a  good  instructor  tool,  resource,  and  reference  through  which  to  glean 
questions,  scenarios,  and  techniques  to  use  during  instruction.  Respondents  also  commented  that 
the  guide  provides  valuable  tools  for  assessing  the  development  of  student  skills  and  attributes. 

Utilization  and  Dissemination  of  Findings: 

The  Instructor’s  P2P  Learning  Guide  for  the  ARC  provides  a  useful  set  of  guidelines  and 
tools  for  realizing  Soldier  skills  and  attributes  and  operationalizing  the  FSTD  approach.  The 
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guide  is  currently  being  used  by  ARC  cadre,  and  its  utilization  and  impact  are  expected  to 
expand  in  stride  with  ARC  implementations  by  the  Maneuver  Center  of  Excellence.  A 
companion  publication  presents  the  complete  Instructor’s  Guide,  which  has  been  posted  on  the 
Army  Training  Network  Web  site  (https://atn.army.mil;  go  to:  products;  training  enablers; 
OBT&E).  By  helping  instructors  train  and  develop  the  skills  and  attributes  needed  by 
reconnaissance  leaders  on  complex  battlefields,  the  Instructor’s  Guide  helps  translate  the 
emerging  FSTD  approach  into  FSO-capable  Soldiers.  The  guide’s  P2P  training  methods  and 
materials  and  its  focus  on  the  outcomes  of  training  and  development  can  be  readily  adapted  to 
foster  the  Soldier  and  leader  skills  and  attributes  required  for  FSO. 
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FULL  SPECTRUM  TRAINING  AND  DEVELOPMENT: 

SOLDIER  SKILLS  AND  ATTRIBUTES 

Introduction 

As  today’s  operational  environments  continue  to  transform.  Soldiers  who  can  adapt  to 
uncertain  and  unforeseen  challenges  have  a  distinct  advantage  (Haskins,  2009a).  The  enemy  is 
no  longer  highly  predictable  as  was  the  case  in  earlier  eras.  Soldiers  are  currently  engaged  with 
an  unpredictable,  adaptive  enemy  and  face  unforeseen  problems  that  require  them  to  be  capable 
problem  solvers.  Historically,  Army  leaders  approached  training  by  conducting  a  mission 
analysis,  generating  a  task  list,  and  gearing  training  around  those  tasks  (Ferguson,  2008). 

Soldiers  were  trained  to  meet  a  minimum  standard  of  performance  on  tasks  which  they  could 
efficiently  execute  in  a  predictable  environment.  This  strategy  is  no  longer  optimal  because 
current  missions  are  increasingly  complex  due  to  unpredictable  situations.  As  Field  Manual  3-0 
(Department  of  the  Army,  2008a)  describes,  Soldiers  now  operate  in  a  full  spectrum  environment 
that  requires  adaptive  thinking  and  careful  use  of  judgment. 

The  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  is  stressing  the  importance 
of  Soldier  attributes  in  conjunction  with  knowledge  and  skills  (Department  of  the  Army,  2009a) 
to  handle  the  uncertainty  of  Full  Spectrum  Operations  (FSO).  “Foundationally,  all  Anny 
training  or  education  integrates  attributes  that  include  Army  values  and  Warrior  ethos.  More 
recently,  in  support  of  FSO,  we  have  begun  to  deliberately  integrate  attributes  such  as 
accountability,  initiative,  confidence,  and  problem-solving”  (Department  of  the  Army,  2009a,  p. 
64).  Particularly  significant  is  the  focus  on  integrating  skills  and  attributes  Soldiers  and  leaders 
need  to  succeed  across  the  full  spectrum  of  military  operations. 

Counterinsurgency  (COIN)  is  essentially  a  human  endeavor  that  taxes  the  full  spectrum 
of  human  capabilities.  The  challenges  of  FSO,  and  particularly  COIN,  demand  an  expanded 
training  approach  that  complements  the  more  comprehensive  concept  of  Soldier  preparation. 

This  report  advocates  an  approach  called  Full  Spectrum  Training  and  Development  (FSTD)  that 
focuses  simultaneously  on  fostering  Soldier  and  leader  skills  and  attributes.  The  FSTD  approach 
emphasizes  the  core  skills  and  attributes  required  to  accomplish  FSO  missions  and  adapt  to  the 
uncertainties  and  complexities  of  COIN.  After  briefly  examining  the  requirement  for  FSTD,  this 
report  documents  the  development  and  evaluation  of  an  instructor’s  guide  focused  on  developing 
the  skills  and  attributes  needed  for  Army  reconnaissance  leaders  in  FSO. 

Vandergriff  (2006)  defines  adaptability  as  “the  process  by  which  individuals  and  groups 
decide  rapidly,  almost  instinctively,  to  changes  in  their  situation”  (p.  43).  A  Soldier’s  experience 
in  problem-solving  exercises  helps  him  become  an  adaptive  leader  (Vandergriff,  2006).  Such 
experience  is  important  in  training  so  that  Soldiers  know  “what  right  looks  like”  (Bard,  2009). 
The  U.S.  Army  has  responded  to  the  need  to  train  and  educate  Soldiers  to  operate  in  a  full 
spectrum  environment.  For  example,  the  Anny  Center  for  Enhanced  Performance  seeks  to 
develop  Soldiers  who  are  self-aware,  instinctive,  adaptive,  and  mentally  agile  (Burbelo,  2009). 
The  Army  must  explore  new  methods  of  training  and  education  to  produce  warfighters  who  can 
think  and  act  more  effectively  in  complex  environments. 
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The  FSTD  approach  demonstrated  here  leverages  the  Army’s  increasing  use  of  two 
training  and  development  methodologies:  peer-to-peer  (P2P)  training  and  outcomes-based 
training  and  education  (OBT&E).  The  objective  of  P2P  training  is  to  build  knowledge,  skills  and 
attributes  through  the  interaction  of  equal-status  individuals,  rather  than  a  traditional  teacher- 
student  paradigm  (e.g.,  Costanza,  Leibrecht,  Cooper,  &  Sanders,  2009;  Topping,  2005).  The 
objective  of  OBT&E  is  to  focus  on  the  results  or  outcomes  of  training  rather  than  the  process  of 
training  (Haskins,  2009b;  Vandergriff,  2009).  One  goal  of  OBT&E  is  to  produce  Soldiers  who 
achieve  skill  proficiency  rather  than  meeting  a  minimum  standard.  A  concurrent  goal  is  to 
develop  attributes  such  as  confidence,  awareness,  sound  decision  making,  leadership,  and  the 
ability  to  apply  their  learning  to  achieve  a  goal.  Both  P2P  and  OBT&E  are  rooted  in  sound 
learning  principles  that  enable  active  learning.  Together,  P2P  and  OBT&E  can  promote  the 
development  of  intangible  attributes,  such  as  problem-solving  and  adaptive  thinking,  in  Soldiers 
who  can  successfully  perform  in  COIN  and  FSO.  By  design,  P2P  training  and  OBT&E  are  key 
methods  for  achieving  the  full  spectrum  of  human  capabilities  advocated  by  FSTD. 

Background 


The  Army  Reconnaissance  Course 

At  Fort  Knox,  Kentucky,  the  Anny  Reconnaissance  Course  (ARC)  is  a  specialty  program 
in  the  vanguard  of  training  transformation.  The  ARC’S  purpose  is  “to  prepare  commissioned 
officers  and  noncommissioned  officers  (NCOs)  to  perfonn  effectively  as  leaders  of  recon 
platoons”  (Perry  &  McEnery,  2009,  p.  14).  The  course  teaches  commissioned  officers  and 
NCOs  the  fundamentals  of  reconnaissance,  surveillance,  and  security  at  the  platoon  level  during 
27  training  days.  The  graduates  of  the  ARC  should  be  capable  of  serving  as  reconnaissance 
leaders.  The  course  utilizes  several  training  techniques  that  include  discussion,  computer-based 
training,  rapid  decision-making  exercises,  virtual  gaming,  and  live  training  scenarios.  The  ARC 
aims  to  develop  adaptive  leader  qualities,  in  addition  to  fundamental  tactical  and  technical  skills, 
in  order  to  fully  prepare  leaders  of  reconnaissance  platoons  for  FSO  (Perry  &  McEnery,  2009). 

The  ARC  program  was  formerly  known  as  the  “Scout  Leader's  Course”  with  a  primary 
focus  on  training  leaders  of  scout  platoons  within  Armor  and  Cavalry  units.  As  an  Army-wide 
program,  the  ARC  has  expanded  to  meet  the  needs  of  all  Armor,  Cavalry,  and  Infantry  units  in 
the  Army.  This  resulted  not  only  in  more  students  trained  per  year,  but  in  a  broadened  scope  of 
the  training.  Previously  the  course  concentrated  on  developing  reconnaissance  knowledge  and 
skills  in  a  Cold  War  setting.  However,  those  knowledge  sets  and  skills  are  no  longer  enough  to 
meet  the  needs  of  today’s  Army  operating  in  a  complex  operational  environment.  Accordingly, 
the  ARC  program  expanded  the  instructional  focus  to  include  training  of  reconnaissance  leader 
attributes.  The  expanded  focus  aims  to  develop  adaptable  and  agile  leaders  able  to  identify 
reconnaissance  and  security  requirements  and  successfully  conduct  all  phases  of  mission 
planning,  preparation,  execution,  and  assessment  in  today’s  operational  environment. 

The  ARC  is  designed  to  meet  the  demands  of  current  reconnaissance  units  while  bridging 
the  gap  between  baseline  institutional  training  and  higher  skill-level  expectations  required  for 
effective  leaders  (Perry  &  McEnery,  2009).  Reconnaissance  leaders  must  be  able  to  generalize 
knowledge  beyond  branch-specific  duties.  They  are  expected  to  have  a  greater  understanding  of 
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the  commander’s  intent  and  how  to  communicate  critical  information,  have  better  planning  and 
execution  skills,  be  more  competent  in  employing  assets,  and  more  confident  in  mission-related 
judgment  than  other  Anny  leaders  (Perry  &  McEnery,  2009).  Rather  than  tailor  several  training 
courses  to  specific  platforms,  the  ARC  has  refined  an  extensive  range  of  doctrinal  tasks  into 
fundamental  skills  that  can  be  applied  across  platforms  (Riccio,  2009).  The  ARC  utilizes  a 
mission-driven,  problem-solving  instructional  framework,  organized  in  a  building  block  manner. 
Skills  and  attributes  are  developed  throughout  each  block  of  training  in  a  challenging  operational 
environment;  most  learning  takes  place  in  the  field  rather  than  the  classroom.  As  training 
progresses,  reconnaissance  leaders  are  challenged  to  apply  their  emerging  competencies  to 
subsequent  training  exercises  in  both  the  classroom  and  the  field.  The  leader-students  are  held 
accountable  for  their  learning  and  progress  in  this  application-focused  course. 

The  ARC  strives  to  increase  students’  problem-solving,  intangibles  (confidence, 
accountability,  initiative,  judgment),  understanding  and  awareness,  and  deliberate  thought  - 
thereby  improving  combat  performance  (Etheridge,  2009;  Riccio,  2009).  In  short,  the  course 
builds  reconnaissance  leader  skills  and  attributes.  Mission-relevant  exercises  allow  instructors  to 
provide  feedback  on  desired  outcomes  so  the  reconnaissance  leaders  come  to  comprehend  why 
they  are  carrying  out  a  mission  in  a  particular  manner.  That  comprehension,  in  turn,  increases 
their  understanding  and  awareness.  While  accomplishing  a  mission,  reconnaissance  leaders  must 
learn  which  information  about  terrain  and  enemy  is  relevant  to  the  commander’s  intent.  They 
must  also  develop  the  skill  set  that  allows  them  to  effectively  communicate  this  information  to 
their  commander.  Soldiers  are  placed  in  situations  where  they  must  develop  solutions  on  their 
own.  Thus,  the  ARC  focuses  on  “how”  to  think  rather  than  “what”  to  think.  Reconnaissance 
leaders  leam  to  engage  in  deliberate  thought  when  under  stress  by  evaluating  the  situation,  using 
sound  judgment,  and  then  making  a  decision. 

Courses  designed  to  train  doctrinal  tasks  may  be  evaluated  by  checking  off  the  completed 
tasks  (Perry  &  McEnery,  2009).  Most  traditional  assessment  tools  employ  a  check-the-box 
method  to  indicate  meeting  a  yes-or-no  standard  (Riccio,  2009).  However,  intangible  attributes 
such  as  confidence  and  initiative,  or  core  competencies  such  as  problem-solving  and  deliberate 
thought,  are  not  adequately  assessed  by  checking  a  box.  Rather  than  relying  on  a  checklist  to 
assess  whether  a  Soldier  has  met  a  requirement,  assessment  should  focus  on  fundamental  skills 
and  attributes  as  continuous  variables,  not  discrete.  Instructors  need  tools  to  aid  them  in 
assessing  FSTD  outcomes. 

The  ARC  program  is  currently  transitioning  from  a  traditional  training  model  to  an 
approach  that  emphasizes  P2P  training  and  OBT&E.  One  of  the  goals  in  moving  to  the  new 
approach  is  to  build  the  attributes  that  reconnaissance  leaders  need  for  FSO.  The  transition  to 
P2P  training  and  OBT&E  methods  makes  the  course  an  ideal  forum  for  developing  innovative 
methods  to  facilitate  FSTD. 

Peer-to-Peer  Training 

The  P2P  training  principles  embody  methods  designed  to  share  knowledge,  skills,  and 
attributes  with  others.  Previous  research  has  shown  that  P2P  training  has  strong  potential  for 
identifying  emerging  lessons  learned  and  integrating  them  into  Army  training  (Costanza  et  ah, 
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2009).  Offering  several  advantages,  P2P  training  is  ideal  for  adult  learners,  supports  the 
generation  of  new  knowledge,  can  complement  traditional  and  nontraditional  forms  of  learning, 
and  accommodates  flexible  scheduling.  Grounded  in  a  sound  scientific  foundation,  P2P  training 
has  roots  in  behavioral,  cognitive,  and  sociocognitive  learning  theories.  In  the  ARC,  P2P 
training  can  guide  cadre  and  student  interaction  while  creating  an  active  learning  environment. 

Recent  efforts  (e.g.,  Clark,  2005;  Costanza  et  al.,  2009;  O’Malley  &  Townsley,  2006) 
have  supported  the  adoption  of  P2P  training  and  a  closely  related  method,  guided  experiential 
learning.  Traditional  instructional  methods  have  been  increasingly  judged  as  inadequate  to  meet 
the  needs  of  complex  training  environments  in  which  Soldiers  must  respond  quickly  to  ever- 
changing  threats  and  requirements.  Soldiers  must  be  able  to  leam  from  each  other  and  rapidly 
apply  their  new  learning. 

Dewey  first  proposed  experiential  learning  in  the  early  1900’s  (Clark,  2005),  but  research 
has  found  that  the  pure  form  of  unstructured  experiential  learning  proposed  by  Dewey  is  not  a 
desirable  means  to  acquire  new  knowledge  because  it  is  time-consuming,  frustrating  to  learners, 
and  inefficient,  among  other  drawbacks.  A  growing  body  of  research,  however,  does  support  the 
utility  of  P2P  training,  or  guided  learning,  which  structures  and  tailors  the  learning  experience  to 
the  needs  of  students  (e.g.,  Clark,  2005;  Gillies,  2007;  Kirschner,  Sweller,  &  Clark,  2006;  Mayer, 
2004;  O’Donnell,  2002). 

The  research  support  for  P2P  training  is  consistent  with  behavioral,  cognitive,  and 
sociocognitive  learning  theories  (Costanza  et  al.,  2009).  Behavioral  theory  prescribes  gradually 
approximating,  or  shaping,  the  desired  response  until  it  meets  criterion.  Cognitive  theory  directs 
the  organization,  or  architecture,  of  learning  by  helping  the  learner  to  associate  newly  learned 
material  with  previously  learned  material  and  by  facilitating  encoding  and  retrieval  processes. 
Sociocognitve  theory  emphasizes  the  effects  of  the  group  context  and  social  interaction  on 
learning.  Teachers  or  knowledgeable  peers  guide  the  student  with  questions  and  structured 
activities.  This  social  interaction  affords  the  chance  to  critique  student  performance  and  helps 
keep  students  engaged  in  the  process.  Sociocognitive  theory  is  also  consistent  with  aspects  of 
social  interdependence  theory,  which  suggests  that  socioemotional  as  well  as  cognitive  benefits 
can  accrue  from  such  training  (O’Donnell,  2002).  This  could  benefit  cohesion  and  other 
desirable  processes  of  team  development. 

A  number  of  advantages  of  P2P  training  have  been  identified  (Costanza  et  al.,  2009), 
such  as  suitability  for  adult  learners,  development  of  new  knowledge,  compatibility  with 
traditional  learning  methods,  and  flexibility  of  scheduling.  The  research  suggests  that  P2P 
training  is  ideally  suited  for  the  modern  combat  environment  in  that  it  can  be  focused  on  the 
identification  of  problems  and  the  generation  and  evaluation  of  solutions  to  problems.  Adaptive 
thinking  is  essential  in  today’s  full  spectrum  operational  environment.  According  to  Woodie 
(2005),  the  Army  should  prepare  Soldiers  to  “unfreeze.”  This  means  the  learners  should  become 
able  to  change  their  current  beliefs  about  the  way  things  are  done  and  view  situations  more 
objectively.  Practical  exercises  and  simulations  help  the  learners  translate  what  they  have 
learned  into  situated  application.  Furthermore,  Soldiers  can  leam  quickly  from  other,  more 
experienced  Soldiers  as  well  as  from  instructors.  Through  P2P  training,  Soldiers  and  their  teams 
or  units  can  become  self-guided  learners. 
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Several  reviews  have  identified  best  practices  that  can  be  used  to  clearly  conceptualize 
and  implement  P2P  training.  For  instance,  Sundstrom  (1999)  described  a  number  of  teaching 
techniques  and  relevant  characteristics  of  teams.  He  noted  that  P2P  training  promotes  students’ 
active  involvement  in  learning  and  that  processing  new  information  at  higher  levels  during 
discussion  helps  organize  students’  experiences  and  facilitate  deeper  understanding.  Positive 
interdependence  among  students  links  individual  success  to  team  success.  Group  processing, 
which  involves  questioning  and  evaluating  decisions,  helps  promote  participation  and  assessment 
of  the  degree  to  which  decisions  are  consensual.  The  problem-based  nature  of  P2P  training 
motivates  students  (Clark,  2005).  Clark  suggests  instructors  should  structure  guided  learning  so 
that  specific  goals  are  linked  to  prior  knowledge.  Careful  monitoring  and  feedback  of  the 
learning  process  occurs  so  that  training  can  be  assessed  and  revised. 

Sundstrom  (1999)  discussed  guidelines  for  team  structure  and  development.  He  noted 
that  teams  should  be  relatively  small  and  that  roles  of  team  members  should  be  specifically 
defined.  Communication  and  feedback  should  be  used  to  keep  team  members  informed  about 
performance.  Leaders  should  help  organize  group  goals  and  assist  team  members  to  take  on 
more  responsibility  and  develop  new  skills.  Further,  the  base  of  knowledge  and  expertise  among 
group  members  should  be  considered  when  forming  the  team.  Less  knowledgeable  members 
should  be  paired  with  more  experienced  team  members,  for  example. 

Costanza  et  ah,  (2009)  defined  P2P  best  practices  as  those  that  were  found  to  be  effective 
across  time  and  situations.  Best  practices  distilled  from  reviewing  available  documentation  and 
interviewing  practitioners  were  used  to  develop  a  P2P  Training  Facilitator’s  Guide  (Costanza  et 
al.,  2009).  The  guide  was  organized  according  to  three  aspects  of  P2P  training  -  development, 
delivery,  and  assessment.  Each  of  these  aspects  covered  a  number  of  the  principles  of  P2P 
training  that  were  derived  from  the  best  practices.  Development,  for  example,  included  activities 
that  facilitators  take  when  preparing  for  training,  such  as  identifying  measureable  training 
objectives,  considering  modes  of  delivery,  and  selecting  group  activities  to  enhance  social 
interaction.  Delivery  encompassed  such  things  as  group  size  and  composition,  techniques  to 
enhance  participation,  the  amount  of  time  taken  to  deliver  the  sessions,  choice  of  questioning 
techniques,  and  consideration  of  group  structure  as  it  relates  to  interaction.  Assessment  was  used 
before,  during,  and  after  training  and  included  specialized  measures  employing  interviews, 
surveys,  observations,  and  self-reports. 

The  final  version  of  the  P2P  Training  Facilitator’s  Guide  was  organized  along  the  five 
basic  principles  of  peer  assisted  learning  (Topping  &  Ehly,  2001)  -  organization  and  structure, 
cognitive  conflict,  scaffolding  and  error  management,  communication  skills,  and  affect.  Each  of 
these  principles  was  linked  to  sub-principles  and  specific  steps  to  be  taken  by  facilitators,  using  a 
decision  tree  framework.  Each  of  the  steps  was  keyed  to  a  particular  section  of  the  guide.  The 
guide  included  practical  exercises  with  problem-based  scenarios  to  give  facilitators  practice  in 
using  the  guide.  Based  on  a  formative  evaluation,  Costanza  et  al.,  (2009)  concluded  that  the 
guide  was  a  useful  product  that  could  be  used  in  future  research  and  development  projects. 

In  summary,  P2P  training  has  been  shown  to  be  an  effective  method  of  training  with  clear 
benefits.  Researchers  and  practitioners  have  successfully  translated  basic  theory  and  principles 
of  learning  and  small  group  interaction  into  an  efficient,  practical  instructional  technology. 
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Outcomes-Based  Training  and  Education 

As  an  instructional  approach,  OBT&E  focuses  on  developing  intangible  attributes,  such 
as  confidence,  self-awareness,  and  deliberate  thought  that  every  Soldier  and  leader  needs  in  a 
rapidly  changing  environment  (Department  of  the  Army,  2008a).  Riccio  (2009)  asserts  that 
OBT&E  is  not  a  teaching  method,  but  a  framework  to  guide  instructional  design.  Soldiers  “learn 
for  themselves”  the  skills  necessary  to  complete  a  mission  within  an  established  framework  of 
knowledge.  The  instructor  teaches  within  the  framework  by  providing  topics  of  conversation  for 
P2P  sharing  of  best  practices,  and  offering  the  “right”  amount  of  guidance  through  worked 
examples  and  explaining  the  assumptions  behind  the  solved  problems. 

Traditional  training  methods  have  limited  ability  to  instill  intangible  attributes  required 
for  FSO.  For  example,  failure  to  understand  the  reasons  for  carrying  out  a  mission  in  a  particular 
manner  limits  Soldiers’  ability  to  “think  on  their  feet”  in  a  slightly  different  situation  (Ferguson, 
2008;  Haskins,  2009a).  In  OBT&E,  tasks  found  in  doctrinal  publications  serve  as  a  foundation 
on  which  mastery  and  excellence  are  built  (Foster,  2009).  Officers  and  NCOs  have  traditionally 
relied  too  heavily  on  the  training  process  (amount  of  time  spent  or  ammo  used)  rather  than  the 
results  (a  competent  Soldier).  Trainers  may  often  have  a  “check-the-box”  mentality,  in  which 
trainees  are  focused  on  simply  meeting  the  minimum  standards.  The  task-conditions-standard 
approach  sufficed  during  the  Cold  War  when  the  enemy  was  predictable.  However,  training 
approaches  that  focus  on  adhering  to  a  set  of  static  rules  may  suppress  initiative,  resulting  in 
Soldiers  who  rely  on  being  told  what  to  do  by  superiors  before  they  take  action  (Comell- 
d’Echert,  2009).  The  current  operational  environment  requires  adaptive  behaviors  to  deal  with 
an  unpredictable  enemy.  Training  that  develops  skills  and  education  that  builds  attributes  are 
both  essential  and  OBT&E  emphasizes  both. 

The  U.S.  Army  has  recognized  the  need  for  new  training  approaches  in  order  to  develop 
Soldiers  who  succeed  in  FSO.  As  Field  Manual  (FM)  7-0  states,  “Traditional  training  and 
education  may  not  meet  all  the  needs  of  an  expeditionary  Anny  . . .  developing  a  new  approach 
may  be  necessary  to  ensure  Soldiers  and  Anny  civilians  are  confident  in  their  ability  to  conduct 
FSO”  (Department  of  the  Army,  2008b,  p.  3-2).  Training  methods  often  do  not  represent  real- 
world  situations  that  require  problem-solving  and  initiative,  nor  is  there  opportunity  for  Soldiers 
to  make  mistakes  and  learn  from  them  (Haskins,  2009b).  The  Army’s  traditional,  input-oriented 
training  methods  need  to  adapt  to  prepare  Soldiers  for  a  full  spectrum  operational  environment 
(Ferguson,  2008). 

There  is  a  need  to  shift  the  leader  training  focus  from  enforcing  standards  to  teaching 
effective  problem-solving  (Haskins,  2009b).  In  any  training  environment,  it  may  not  be  possible 
to  anticipate  the  types  of  problems  leaders  could  encounter.  Therefore,  training  leaders  how  to 
succeed  in  almost  any  situation  by  developing  intangible  attributes  is  important.  It  is  argued  that 
OBT&E  is  an  effective  methodology  for  building  intangible  attributes.  At  the  U.S.  Military 
Academy  (Haskins,  2009b),  OBT&E  focuses  on  achieving  desired  outcomes,  such  as  sound 
decision-making  and  judgment.  Rather  than  “checking-the-box”  to  make  sure  leaders  attain 
minimum  training  standards,  OBT&E  can  produce  leaders  who  are  motivated  to  reach  a  higher 
level  of  performance  (Tice,  2008).  The  goal  of  OBT&E  is  to  achieve  competence  in  a  task  or 
skill  rather  than  meet  the  minimum  standard  or  to  reach  an  “adequate”  performance  level. 
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Recognition  that  skill-based  competence  requires  both  cognitive  understanding  and  actual 
performance  is  one  of  the  underpinnings  of  OBT&E  (Dennis,  2009).  Soldiers  have  reasonable 
autonomy  during  training  in  order  to  facilitate  becoming  competent  and  confident  in  their 
abilities.  The  OBT&E  principles  include  (a)  enhancing  problem-solving  ability,  (b)  developing 
intangible  attributes,  (c)  increasing  understanding  and  awareness,  (d)  increasing  deliberate 
thought,  and  (e)  improving  combat  performance.  The  OBT&E  approach  teaches  the  basics,  but 
also  focuses  on  mastery  and  full  understanding  of  basic  skill  sets,  resulting  in  Soldiers  who  can 
improvise  and  adapt  their  knowledge  to  solve  problems  in  varied  situations  (Connolly,  2008; 
Ferguson,  2008).  A  “crawl,  walk,  run”  approach  enables  Soldiers  to  develop  confidence  and 
other  intangible  attributes  (Connolly,  2008). 

The  use  of  real  world  exercises  that  guide  Soldiers  to  focus  on  the  “why”  rather  than 
merely  the  “what”  and  “how”  differentiates  OBT&E  from  traditional  training  methods.  Also, 
OBT&E  trainers  “coach”  rather  than  “direct,”  and  they  “develop”  rather  than  “instruct”  (Cornell- 
d’Echert,  2009).  Other  characteristics  of  OBT&E  include  ensuring  teachers  fully  explain 
concepts  to  Soldiers,  creating  an  environment  where  making  mistakes  is  acceptable,  and 
providing  constant  feedback  to  Soldiers  about  their  actions.  Training  facilitators  use  Socratic 
questioning  to  prompt  Soldiers  to  clarify  what  they  are  already  thinking  about  or  to  think  more 
analytically  so  they  can  work  out  the  problem.  Socratic  questioning  can  also  serve  to  investigate 
assumptions,  rationale  for  arguments,  consequences  or  implications  of  an  action,  and  alternative 
viewpoints  (Straker,  2009).  Examples  of  Socratic  questions  include:  “Are  these  reasons  good 
enough?”  “What  evidence  do  you  have  to  support  this  assumption?”  “How  could  you  look  at  it 
another  way?”  And,  “What  would  happen  if. . .?”  The  following  is  an  example  application  of 
OBT&E  principles  from  the  Fires  Center  of  Excellence  at  Fort  Sill. 

“When  zeroing  a  M16  rifle,  a  Soldier  would  typically  fire  at  a  paper  target  from  a  25- 
meter  range.  The  targets  include  standard  written  instructions  that  a  Soldier  can  read  to 
tell  him  how  many  clicks  left  or  right  to  move  for  the  optimal  firing  position.  A  Soldier 
may  adjust  his/her  position  according  to  the  instructions  without  understanding  why.  In 
an  OBT&E  approach,  Soldiers  leam  about  angular  deviation  and  ‘the  function  of  the 
weapon’s  front  sight  post  and  what  fraction  of  a  MOA  [minutes  of  angle]  each  click,  left 
or  right,  actually  represents’”  (Dennis,  2009,  p.33). 

In  this  example,  the  knowledge  gained  from  OBT&E  training  enables  Soldiers  to  use 
direct  visual  feedback  to  fine-tune  the  zeroing  of  the  weapon  on  their  own  (i.e.,  solve  a  simple 
problem). 

The  results  of  using  OBT&E  as  a  training  method  include  Soldiers  who  teach  themselves, 
solve  problems  based  on  knowledge  and  principles,  and  possess  tangible  skills  as  well  as 
intangible  attributes.  Although  an  intangible  attribute  cannot  be  physically  measured,  the 
behavior  that  intangibles  are  likely  to  influence  can  be  measured  (Marceau,  Diedrich  &  Riccio, 
2008).  It  is  important  to  continually  assess  the  OBT&E  process  during  training.  Continuous 
assessment  provides  feedback  on  Soldiers’  progress  on  achieving  the  desired  outcome  and 
shapes  how  the  teacher  will  adjust  training,  if  needed  (Haskins,  2009b). 
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Examples  of  successful  application  of  OBT&E  in  current  training  environments  include 
West  Point’s  MS300  curriculum  in  the  Department  of  Military  Instruction,  the  Armor  School’s 
ARC,  and  the  Combat  Applications  Training  Course  (CATC)  (Comell-d’Echert,  2009).  The 
CATC  demonstrates  OBT&E  methodology  using  rifle  marksmanship  tasks  to  lead  Soldiers  in 
building  intangible  attributes  such  as  confidence  and  motivation  (Connolly,  2008).  Using  an 
outcome-based  mindset,  in  conjunction  with  marksmanship  training,  gives  Soldiers  greater 
understanding  of  how  and  why  their  weapon  works  as  it  does  and  instills  confidence  when  using 
their  weapon  in  an  unpredictable  situation  (Ferguson,  2008). 

In  addition  to  building  basic  competence,  the  OBT&E  approach  also  creates  opportunities 
for  Soldiers  to  use  what  they  have  learned  with  reasonable  autonomy  and  flexibility  in  a  practical 
training  situation  (Department  of  the  Army,  2009b).  This  training  approach  results  in  the  ability 
to  apply  knowledge  and  skills  to  varied  operational  environments.  Instructors  should  reinforce 
why  tasks  are  carried  out  a  certain  way  to  support  overall  mission  success.  Rather  than  the 
instructor  telling  Soldiers  what  to  do,  the  OBT&E  approach  guides  Soldiers  to  a  solution  using 
problem-solving  strategies.  This  action  provides  Soldiers  with  a  sense  of  ownership  and 
empowerment.  Throughout  training,  instructors  create  opportunities  for  Soldiers  to  develop  a 
sense  of  accountability  by  assuming  responsibility  for  their  actions. 

Based  on  the  foregoing  discussion,  the  major  features  and  characteristics  of  OBT&E  are 
summarized  as  follows: 

•  OBT&E  is  a  method  of  force  preparation  that  seeks  to  merge  the  benefits  of  training 
and  education  to  create  thinking  leaders  who  know  what  to  do  in  the  complex 
environment  of  FSO. 

•  The  guiding  principles  of  OBT&E  are  to  grow  problem-solving,  increase  intangibles, 
boost  understanding  and  awareness,  increase  deliberate  thought,  and  improve  combat 
performance. 

•  Intangible  attributes — including  confidence,  accountability  and  initiative — underpin 
Soldier  development  and  operational  performance. 

•  The  desired  outcomes  for  OBT&E  are  Soldiers,  leaders,  and  units  who  know  how  to 
teach  themselves.  They  solve  problems  as  individuals  and  teams  using  knowledge  and 
principles.  They  are  empowered  by  their  mastery  of  tangible  skills  and  their  intangible 
attributes. 

•  The  basic  themes  of  OBT&E  stress  understanding  and  mastery  of  basic  skills, 
development  of  intangible  Soldier  attributes,  and  ability  to  relate  tasks,  skills,  and 
attributes  to  mission  objectives. 

•  OBT&E  training  emphasizes  the  total  outcome  of  a  task  and  training  event,  rather  than 
the  execution  of  a  particular  task  to  a  standard  under  a  given  set  of  conditions. 

Because  training  agile  and  adaptive  leaders  is  critical  for  Army  reconnaissance  units, 
applying  OBT&E  to  the  ARC  is  especially  warranted.  The  transformed  program  stresses  (a) 
mastery  of  fundamental  reconnaissance  skills  so  leaders  can  improvise  and  adapt  knowledge  in 
varied  situations,  (b)  training  through  mission  relevant  exercises  that  focus  on  developing 
intangible  attributes,  and  (c)  encouraging  thinking  in  terms  of  missions  and  problem-solving  by 
developing  the  ability  to  relate  knowledge  and  skills  to  other  tasks. 
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Research  Objectives 


The  research  described  in  this  report  was  perfonned  as  collaboration  between  the  U.S. 
Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  and  the  ARC  cadre  at  Fort 
Knox.  The  project  aimed  to  develop  innovative  methods  to  enhance  the  ARC  instructional 
process  by  creating  practical  materials  to  help  the  cadre  ensure  the  achievement  of  desired  course 
outcomes.  There  is  a  clear  need  for  innovative  methods  because  a  core  component  of  FSTD  is 
the  development  and  assessment  of  key  attributes  (confidence,  initiative,  judgment, 
accountability,  etc.).  Traditional  methods  of  instruction  and  assessment  are  largely  ill-equipped 
to  address  such  intangibles.  The  following  research  objectives  guided  the  work: 

•  Observe  and  coordinate  with  ongoing  efforts  to  apply  P2P  training  to  ARC  activities. 

•  Develop  P2P-type  training  methods  to  improve  the  ability  of  ARC  instructors  to  model 
and  assess  reconnaissance  leader  attributes  and  skills. 

•  Conduct  formative  evaluation  of  P2P  training  and  measurement  methods  by 
implementing  them  operationally  and  gathering  evaluative  feedback. 

•  Package  the  P2P  training  and  assessment  methods  in  an  instructor-friendly  guide. 

•  Document  the  results  of  the  research. 

This  report  describes  the  methods  used  to  accomplish  the  research  objectives  as  well  as 
the  results  of  the  formative  evaluation  efforts.  A  companion  publication  (Cooper,  Leibrecht,  & 
Lickteig,  in  preparation)  presents  the  Instructor’s  P2P  Learning  Guide  for  the  ARC  that  resulted 
from  the  development  of  P2P  training  and  assessment  materials  for  the  ARC  environment. 

Method 

The  purpose  of  this  research  was  to  develop  and  evaluate  an  approach  to  Soldier 
preparation  focused  on  realizing  the  skills  and  attributes  needed  for  reconnaissance  leaders  in 
FSO.  To  that  end,  the  research  team  developed  a  guide  to  assist  ARC  instructors  in  achieving 
course  goals  by  incorporating  P2P  training  principles.  The  approach  was  to  indentify  the  core 
outcomes  of  the  ARC  and  identify  opportunities  where  P2P  training  principles  could  be  applied 
to  achieve  those  goals.  The  research  began  with  a  literature  review  of  the  best  practices  from 
industry,  academia,  and  the  military.  Educational  materials,  procedural  guidelines,  and  job  aids 
were  developed  based  upon  the  literature  review  and  observational  data  collected  during  pilot 
courses.  The  resulting  guide  underwent  operational  implementation  and  multiple-stage,  multi¬ 
source  formative  evaluation. 


Literature  Review 

The  literature  review  was  performed  to  support  the  development  of  an  ARC  instructor's 
guide  for  conducting  P2P  training.  The  goal  of  the  review  was  to  find  P2P  methods  suitable  for 
the  ARC  program  to  assist  instructors  in  training  and  measuring  student  outcomes.  The  review 
identified  best  practices  for  P2P  learning,  best  practices  in  OBT&E  training,  and  assessment 
techniques  for  intangible  attributes  and  learning  outcomes.  The  relevance  of  source  materials 
was  determined  by  the  researchers  according  to  how  well  each  reference  supported  the  goals  of 
ARC,  P2P  training,  OBT&E,  and  construction  of  the  instructor’s  guide.  Articles  and  books  were 
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selected  from  academic,  industry,  and  military  sources  based  on  the  relevance  to  FSTD,  P2P,  and 
OBT&E.  Materials  were  screened  based  on  their  citation  rates  and  scientific  rigor.  Articles  with 
higher  citation  rates  were  chosen  over  articles  with  low  citation  rates  (as  obtained  through 
GALILEO,  the  search  engine  of  the  University  System  of  Georgia).  Books  and  articles  that 
provided  or  cited  empirical  evidence  and  meta-analyses  were  chosen  over  non-empirical  items. 

The  literature  review  yielded  P2P  and  OBT&E  learning  principles,  best  practices  for 
training  and  assessment,  practical  techniques,  theoretical  considerations,  and  enabling  factors. 
The  results  fed  the  design  and  development  of  the  instructor’s  P2P  learning  guide.  The  more 
notable  findings  were  integrated  into  the  Background  section  of  the  preceding  chapter. 

Development  of  the  Guide 

Interviews  with  OBT&E  advisors,  ARC  leaders  and  instructors,  and  research  team 
experts  occurred  prior  to,  during,  and  following  multiple  ARC  cycles.  The  interview  data  helped 
identify  practical  needs  of  the  ARC  cadre.  They  also  helped  tailor  parts  of  the  guide  to  the  ARC 
program,  especially  the  definition  of  reconnaissance  leader  skills  and  attributes  and  the 
assessment  materials. 

To  set  the  stage  for  designing  and  developing  an  instructor’s  guide,  observations  of  ARC 
activities  were  conducted.  Based  on  the  literature  review,  an  observation  protocol  was 
constructed.  Observational  data  were  gathered  from  multiple  sources.  Researchers  shadowed 
and  interviewed  ARC  instructors  during  multiple  field  exercises  throughout  several  ARC  cycles. 
Researchers  also  observed  classroom  sessions.  The  observational  data  helped  the  developers  of 
the  guide  identify  opportunities  for  P2P  training-based  instruction  in  the  ARC  as  well  as  threats 
to  learning. 

Based  on  an  informal  needs  analysis,  development  of  materials  for  the  guide  centered  on 
empowering  cadre  members  to  foster  reconnaissance  leader  skills  and  attributes.  Topics  were 
selected  and  materials  were  developed  according  to  the  ARC  instructional  objectives  and 
outcomes,  P2P  training  best  practices,  and  OBT&E  principles.  The  P2P  learning  model  for  the 
ARC  was  structured  in  three  phases  -  plan/prepare,  execute,  and  assess  (see  Figure  1)  -  which 
mirror  the  develop-deliver-assess  structure  of  the  P2P  training  model  (Costanza  et  al.,  2009). 
Development  of  the  P2P  training  materials  drew  on  multiple  sources  of  information  including  the 
literature  review  findings,  interviews  with  key  ARC  personnel,  information  gathered  from 
conferences  and  workshops,  and  field  observations  during  ARC  pilot  courses.  The  development 
process  followed  an  iterative  draft-review-revise  cycle. 
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ARC  P2P  Learning  Model 


Figure  1.  Three-stage  P2P  learning  model  (from  Instructor’s  Guide). 

The  ARC  program  goals  and  structure  served  as  key  anchors  for  developing  the  guide. 
All  materials  were  developed  with  a  focus  on  guidelines  and  tools  to  build  reconnaissance  leader 
skills  and  attributes.  Graduates  of  the  ARC  are  expected  to  demonstrate  an  observably  higher 
level  of  proficiency  on  the  reconnaissance  skill  set.  They  should  exhibit  a  deeper  understanding 
of  the  commander’s  intent  and  information  requirements.  They  should  be  more  competent 
problem  solvers  and  display  critical  leadership  attributes.  Figure  2  shows  the  reconnaissance 
leader  skills  and  attributes  as  they  ultimately  appeared  in  the  Instructor’s  Guide. 
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ARC  Recon  Leader  Skills  and  Attributes 

(Applied  Descriptions) 


Recon  Leader  Skills 


Navigation,  Communication  &  Reporting:  exceeds 

Sbtanda'dsfSTP  21-24-  SMC  i  Wamor  Leader 
s.  Levels  2,3 ,4.  September 2006) 

Tactical  Assessment:  assesses  terran,  enemy  and 
fnendy  capabiKies,  and  the  mission  smutaneously 

Understanding  Cdr's  Recon  Needs:  determines 
information  required  for  Commander  decisions 

Acquire  Priority  Targets :  obtains  the  information 
required  for  Commander  decisions 

Decision-Making:  implements  timely  decisions  to 
accompksh  rmsion  requirements  wunrn  constraints  of 
terrain,  enemy  capability  and  fnerxJy  capability 

Communicate  Relevant  Info:  communcates 
observations  in  a  manner  relevant  to  Commander's 
decision-making  needs 

Maintain  Mission  Viability:  antedates  consequences 
of  lacteal  decisions 

Employ  Support  Assets:  demonstrates  understanding 
of  urat/system  supporting  ranges  and  distances,  employs 
techncal  surveillance  tools  effectively,  demonstrates 
ability  to  employ  air  and  ground  based  fires  effectively 


ProblemSolving:  solves  problems  by  applyng 
deliberate  thought 

Anticipation :  foresees  future  requirements  and 
condtions 

Initiative:  thnks  and  acts  wthout  being  urged 

Adaptability:  manages chanang  requirements 
for  balancing  unt  recon,  surveillance,  and 
security  with  mission  accomplishment 

Accountability:  takes  responsibility  for  own 
and  team's  actions  and  consequences 

Confidence:  believes  n  own  and  team's  aWity 
to  hande  tactcal  situations 

Risk  Management:  assesses  the  situation 
aganst  the  mission  and  makes  a  decision  - 
effectively  balances  rrsssion  requirements  and 
nsk 


Figure  2.  Definitions  of  recon  leader  skills  and  attributes  (from  Instructor’s  Guide). 


Because  ARC  training  utilizes  OBT&E  principles,  the  guide  was  developed  to  assist 
instructors  in  applying  those  principles.  The  OBT&E  principles  break  away  from  traditional 
instructor-student  roles  -  instructors  guide  rather  than  direct  the  students.  The  principles  rely 
heavily  on  the  Socratic  and  P2P  methods  of  instruction  where  instructors  ask  leading  questions  to 
allow  students  to  explore  and  identify  the  solutions.  Students  share  knowledge  and  work 
together  to  produce  solutions.  Since  this  method  of  instruction  is  markedly  different  from  the 
minimum  standards  “check-the-box”  approach  the  Army  has  typically  used,  the  guide  was 
shaped  to  clarify  unique  aspects  of  OBT&E  training. 


A  significant  contribution  toward  assisting  instructors  with  implementing  OBT&E 
methods  was  the  inclusion  of  P2P  training  principles.  The  P2P  training  focus  on  peer-assisted 
learning  was  consistent  with  the  first  principle  of  OBT&E  that  calls  for  training  to  grow 
problem-solving  by  teaching  Soldiers  to  “learn  for  themselves.”  Paralleling  this  was  an 
emphasis  on  the  acquisition  of  knowledge,  skills  and  attributes  through  active  sharing,  helping 
and  supporting  among  equals.  Thus,  P2P  training  principles  provided  a  seamless  transition 
between  the  principles  of  OBT&E  and  sound  instructorship.  The  P2P  training  was  intended  to 
enable  students  to  leam  quickly  and  readily  from  instructors  and  other  students  who  have  gained 
experience  and  insight  in  conducting  FSO.  Previous  research  has  indicated  that  P2P  training 
occurs  best  when  facilitated,  with  novices  guided  through  a  complex  task  by  a  facilitator  who 
models  the  processes  and  behaviors  of  effective  learning  (Costanza  et  ah,  2009). 
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The  P2P  training  principles  were  incorporated  throughout  the  guide,  including  its  job 
aids.  For  example,  Figure  3  shows  a  job  aid  that  provides  examples  to  help  instructors  recognize 
P2P  training  opportunities.  The  job  aids  contained  in  the  Instructor’s  Guide  also  include  forms 
to  guide  and  assist  instructors  in  assessing  and  recording  skill  and  attribute  development  among 
students  (see  Figures  4  and  5).  This  fulfilled  the  TRADOC  requirement  that  instruction  must  be 
assessed  (Department  of  the  Army,  2009a).  The  latter  job  aids  structure  assessment  without 
limiting  the  process  to  a  minimum  standards  “check-the-box”  approach.  The  growth  of 
reconnaissance  leader  skills  and  attributes  can  be  monitored  throughout  the  course  to  provide 
students  with  specific  feedback. 


Recognizing  P2P  Training  Opportunities 


JobAidJ-1 


P2P  opportunities  in  ARC  are  dynamic.  Recognizing  a  P2P  opportunity  requires  the  instructor  to 
continuously  monitor  the  student  situation  for  potential  learning  shortfalls.  Below  are  examples 
of  "trigger"  conditions  indicating  likely  P2P  learning  opportunities. 


<<  If  something  is  not  right  -  get  involved  >> 


Opportunities  for  Instructor  Guidance 

Recon  Leader  Skills 

Recon  Leader  Attributes 

•  Do  reports  ignore  CCIR? 

•  Are  reports  unusable  by  higher 
headquarters? 

•  Is  the  unit  at  nsk  for  compromise? 

•  Has  the  unit  lost  freedom  of  maneuver? 

•  Are  survivability  measures  inadequate? 

•  Are  advanced  land  navigation  skills  missing? 

•  Is  the  student  overcome  by  psychological  or 
physical  requirements? 

•  Are  asset  capabilities  being  under-utilized? 

•  Is  local  security  being  compromised? 

•  Does  the  student  fail  to  demonstrate  skills 
necessary  for  mission  success? 

•  Do  student  actions  indicate  a  lapse  in  deliberate 
thinking  (evaluation,  judgment,  decision)? 

•  Do  students  display  a  lack  of  supporting 
knowledge? 

•  Do  students  fail  to  demonstrate  critical  attnbutes 
like  confidence,  accountability,  initiative, 
judgment,  and  awareness? 

•  Are  students  disagreeing  or  arguing  strongly? 

•  Are  one  or  two  students  overpowering  the  rest? 

•  Do  students  seem  confused  or  disoriented? 

•  Have  some  students  become  silent  or 
disinterested? 

•  Are  students  maintaining  100M  IPB  awareness? 

Figure  3.  Job  aid  for  recognizing  P2P  training  opportunities  (from  Instructor’s  Guide). 
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Student  Assessment  (1  of  3) 

(Instructor's  Rating  of  Student  Skills) 


Job  Aid  J-6.1 


Recon  Leader 
Skills 

Scale  Value  1  -  Indicators 

Assessment 
(circle  one) 

* 

Scale  Value  5  -  Indicators 

Navigation 

Never  meets  Army  standards 

1 

2 

3 

4 

5 

Always  exceeds  Army  standards 

Communication  & 
Reporting 

Never  meets  Army  standards 

1 

2 

3 

4 

5 

Always  exceeds  Army  standards 

Tactical  Assessment 

Never  adjusts  for  terrain, 
enemy/friendly  capabilities 

1 

2 

3 

4 

5 

Always  incorporates  terrain, 
enemy/friendly  capabilities 

Understanding  Cdr’s 
Recon  Needs 

Never  identifies  Cdr's  unique  recon 
needs 

1 

2 

3 

4 

5 

Always  identifies  Cdr’s  unique  recon  needs 

Acquiring  Priority 
Targets 

NeveracquiresBCT  priority  targets 

1 

2 

3 

4 

5 

Always  acquires  BCT  priority  targets 

Decision-Making 

Never  makes  sound  decisions  in 
timely  manner 

1 

2 

3 

4 

5 

Always  makes  sound  decisions  in  timely 
manner 

Communicating 
Relevant  Info 

Never  reports  info  critical  to  Cdr’s 
decision-making 

1 

2 

3 

4 

5 

Always  reports  info  critical  to  Cdr's 
decision-making 

Maintaining  Mission 
Viability 

Never  anticipates  presence  or 
contact  with  enemy 

1 

2 

3 

4 

5 

Always  anticipates  presence  or  contact 
with  enemy 

Employing  Support 
Assets 

Neveremploys  supporting  assets 

1 

2 

3 

4 

5 

Always  employs  supporting  assets 
appropriately 

Instructions: 

•  Quantify  the  student's  performance  by  circling  a 
rating  value  (1-5)  for  each  recon  leader  skill. 

•  Discuss  steps  to  improve  within  each  area  of 
assessment  in  the  future. 


*  Rating  Scale: 

1  -  Minimal:  Shows  minimal  level  expected  of  a  novice 

2  -  Moderate:  Shows  moderate  level  expected  of  a  beginning  leader 

3  -  Average:  Meets  Army  standards 

4  -  Superior:  Shows  superior  level  expected  of  a  seasoned  leader 

5  -  Exceptional:  Shows  exceptional  level  expected  of  a  recon  leader 


Figure  4.  Instructor’s  form  for  rating  student  skills  (from  Instructor’s  Guide). 
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Job  Aid  J-6.2 


Student  Assessment  (2  of  3) 

(Instructor's  Rating  of  Student  Attributes) 


Recon  Leader 
Attributes 

Scale  Value  1  -  Indicators 

Assessment* 

Scale Value5  -  Indicators 

Problem 

Solving 

Overlooks  opportunities  to  identify 
and  solve  problems 

1  2  3  4  5 

Leads  team  through  problem  ID,  idea 
generation,  decision-making,  and  feedback 

Anticipation 

Is  surprised  by  how  events 
unfolded 

1  2  3  4  5 

Anticipates  and  is  prepared  for  what’s  next 

Initiative 

Asks  the  instructor  what  to  do 

1  2  3  4  5 

Seeks  solutions  and  solves  problems  in-stride 
using  all  available  assets 

Adaptability 

Changing  conditions  cause  mission 
failure 

1  2  3  4  5 

Adapts  to  changing  conditions  without  mission 
degradation 

Accountability 

Spends  majority  of  time  performing 
tasks  not  essential  to  the  mission 

1  2  3  4  5 

Identifies  and  works  on  tasks  essential  to  the 
mission 

Confidence 

Does  not  interactwith  other 
students 

1  2  3  4  5 

Actively  participates  throughouttraining - 
shares  recon  skills  and  experiences 

Risk 

Management 

Never  balances  mission 
requirements  with  risk 

1  2  3  4  5 

Always  balances  mission  requirements  and  risk 

Instructions: 

•  Quantify  the  student's  performance  by  circling  a  rating 
value  (1-5)  for  each  recon  leader  attribute. 

•  Discuss  steps  to  improve  within  each  area  of  assessment 
in  the  future. 


*  Rating  Scale: 

1  -  Minimal:  Shows  minimal  level  expected  of  a  novice 

2  -  Moderate:  Shows  moderate  level  expected  of  a  beginning  leader 

3  -Average:  Meets  Army  standards 

4  -  Superior:  Shows  superior  level  expected  of  a  seasoned  leader 

5  -  Exceptional:  Shows  exceptional  level  expected  of  a  recon  leader 


Figure  5.  Instructor’s  form  for  rating  student  attributes  (from  Instructor’s  Guide). 


As  the  training  and  assessment  materials  emerged,  the  researchers  and  cadre  leaders 
designed  the  structure  and  format  for  the  Instructor’s  Guide  itself.  Based  on  the  ARC  learning 
environment,  the  stakeholders  designed  the  guide  to  serve  as  an  easy-to-use  resource  for  self- 
study  and  application.  Primary  design  criteria  included  (a)  minimum  essential  contents,  (b) 
balanced  mix  of  self-development  and  job  aid  items,  (c)  embedded  motivation,  (d)  streamlined 
presentation  of  contents,  (e)  liberal  use  of  charts  and  graphics,  (f)  Soldier-friendly  packaging,  (g) 
stand-alone  capability,  and  (h)  ready  portability.  Analysis  and  sorting  of  the  instructors’ 
information  requirements  resulted  in  ten  major  sections  of  the  guide  (topics)  as  seen  in  Figure  6. 
In  addition,  a  front-end  P2P  utilization  scenario  was  included  to  illustrate  how  the  guide  could  be 
employed  in  a  tactical  context. 


The  topics  addressed  in  the  guide  fell  into  three  categories:  educational  information  (e.g., 
P2P  training  basics),  procedural  guidelines  (steps  and  techniques),  and  job  aids  containing  how¬ 
to  details.  The  final  topics  (see  Figure  6)  were  those  found  to  contribute  directly  to  the 
performance  of  ARC  instructional  duties.  The  limited  scope  of  the  contents  was  designed  to 
ensure  a  level  of  detail  suitable  for  rapidly  assimilating  the  information. 
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This  Guide  Contains ... 


Topic 

What’s  Inside 

Page  # 

Guide  Purpose 

How  this  guide  supports  ARC  instructors 

4 

P2P  Training  Basics 

What  is  P2P,  its  role  in  OBTE,  how  it  works 

5 

ARC  P2P  Learning  Process 

Plan/Prepare  -  Execute  -  Assess  model  of  learning 

6 

P2P  Best  Practices 

ARC  instructor  best  practices  for  employing  P2P 

7 

Setting  P2P  Conditions 

Shaping  the  environment  for  P2P  learning 

8 

Recon  Leader  Skills  and 
Attributes 

Definitions  and  Descriptions 

9 

P2P  Training  Techniques 

Suggested  techniques  for  collaborative  learning 

10 

P2P  Assessment  Guidelines 

Best  practices,  assessment  techniques,  timing 

11 

P2P  Assessment  Techniques 

Suggested  steps  for  each  assessment  technique 

12 

Job  Aids  for  Instructors 

Recognizing  P2P  Training  Opportunities  (J-1) 

13 

Using  Guiding  Questions  (J-2) 

14 

Exploiting  P2P  Potential  (J-3) 

15 

Using  Empowering  Statements  (J-4) 

21 

Deterring  Threats  to  Training  (J-5) 

22 

Student  Assessments  (J-6) 

23 

InstructorAssessment  (J-7) 

26 

Hotwash/AAR  Guide  (J-8) 

27 

Figure  6.  Major  sections  of  Instructor’s  Guide. 


Formative  Evaluation 

The  goal  of  the  formative  evaluation  was  to  obtain  user  feedback  based  on  operational 
use  of  the  guide.  The  evaluation  strategy  employed  an  implement-assess-refine  cycle  to  create 
recurring  feedback  opportunities.  The  evaluation  helped  ensure  the  guide  and  its  job  aids  were 
acceptable,  suitable,  usable,  and  valuable  to  leaders  and  instructors  of  the  ARC  program. 

Formative  Evaluation  Events 

Formative  evaluation  occurred  in  three  broadly  defined  stages.  During  the  first  stage,  the 
guide  was  extensively  vetted  prior  to  field  testing  in  available  ARC  cycles.  The  vetting  method 
employed  an  iterative  process  using  multiple  reviewers,  each  providing  feedback  on  multiple 
occasions.  The  review  process  included  a  multi-source  formative  evaluation,  with  reviews  being 
elicited  from  personnel  on  all  levels  of  program  involvement. 

Academic  experts  first  reviewed  the  guide,  including  the  job  aids.  Scientists  within  ARI 
vetted  the  guide  next,  followed  by  the  course  designer,  course  leaders,  and  instructors.  The 
guide  was  vetted  to  ensure  sound  learning  principles  and  outcome  assessments  were  effectively 
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incorporated,  and  that  the  guide  matched  the  goals  and  outcomes  of  the  ARC.  In  addition, 
feedback  was  solicited  on  which  sections  could  be  clarified,  expanded,  reduced,  or  deleted.  The 
vetting  procedure  was  performed  in  multiple  cycles,  with  each  group  given  multiple 
opportunities  to  evaluate  the  guide. 

In  stage  two,  initial  field  testing  of  the  guide  occurred  during  an  ARC  pilot  course. 

During  the  remainder  of  this  report,  this  stage  will  be  referred  to  as  the  first  evaluation.  As  part 
of  the  first  evaluation,  course  designers,  course  leaders,  and  instructors  were  interviewed  prior  to, 
during,  and  after  the  pilot  course  and  asked  to  provide  evaluative  feedback.  After  the  pilot 
course  ended,  the  guide  was  revised  based  upon  the  feedback. 

During  stage  three  of  the  formative  evaluation,  the  final  field  testing  of  the  guide  took 
place  during  a  second  ARC  cycle,  following  the  same  method  and  procedures  used  in  stage  two. 
This  final  stage  will  be  referred  to  as  the  second  evaluation. 

Participants 

Participants  included  course  designers,  course  leaders,  and  instructors.  Course  leaders 
and  instructors  were  interviewed  during  the  material  development,  and  they  also  provided  survey 
feedback  based  on  their  use  of  the  guide  during  the  first  and  second  evaluations.  A  total  of  19 
respondents  ( 1 8  instructors,  1  course  leader)  provided  feedback  during  the  first  evaluation  and  20 
respondents  (19  instructors,  1  course  leader)  during  the  second  evaluation. 

Data  Collection  Materials 

The  research  team  constructed  several  data  collection  instruments  for  the  formative 
evaluation.  These  instruments  incorporated  a  combination  of  qualitative  data  (written  and  verbal 
comments)  and  quantitative  data.  The  data  collection  materials  included: 

•  Observation  Guide  (see  Appendix  B). 

•  ARC  Instructor  Feedback  Questionnaire  (see  Appendix  C). 

•  ARC  Leader  Feedback  Questionnaire  (see  Appendix  D). 

•  Query  Guide  -  End-of-Block  Feedback  Session  (see  Appendix  E). 

Procedure 

Individual  interviews  were  conducted  with  course  leaders  and  instructors  prior  to,  during, 
and  following  courses.  At  the  end  of  each  course,  questionnaire  data  were  collected  from  the 
participants  as  a  group.  Respondents  sat  at  individual  desks  and  completed  the  appropriate 
feedback  questionnaire  according  to  their  role.  Following  the  written  feedback,  a  facilitator 
elicited  verbal  feedback  from  the  respondents  using  the  query  guide  to  structure  the  discussion. 


17 


Results  and  Discussion 


In  the  formative  evaluation  of  the  guide,  the  ARC  cadre’s  feedback  served  as  the  primary 
data  and  provided  valuable  information  to  direct  refinement  efforts.  The  feedback  gathered 
during  each  stage  of  the  evaluation  was  generally  positive.  Comments  and  suggestions  for 
revisions  were  generally  minor  and  were  incorporated  into  the  guide  following  each  stage  of 
evaluation.  The  results  will  be  organized  according  to  four  broad  categories:  acceptability  of  the 
guide,  utility  of  the  guide,  suggested  improvements  to  the  guide,  and  lessons  learned. 

Acceptability  of  the  Guide 

Components  of  the  Instructor’s  Guide  were  evaluated  through  two  sets  of  questions 
addressing  acceptability:  (a)  clarity  of  the  educational  and  guideline  materials  and  (b)  clarity  and 
suitability  of  the  job  aids.  Each  set  of  questions  contained  between  14  and  18  statements  about 
specific  aspects  of  each  part  of  the  guide.  Instructors  and  course  leaders  responded  using  5 -point 
scales  (1 -Strongly  Disagree  to  5-Strongly  Agree),  with  higher  scores  indicating  more  positive 
ratings  unless  otherwise  noted. 

The  educational  and  guideline  materials  of  the  guide  comprised  nine  pages  addressing 
how  to  use  the  guide,  reconnaissance  leadership  skills  and  attributes,  and  P2P  training  best 
practices  and  techniques  to  support  training  and  assessment.  The  clarity  of  these  materials  was 
evaluated  through  eight  written  statements  (first  evaluation)  and  nine  written  statements  (second 
evaluation)  addressing  the  various  components  of  the  materials.  Representative  statements  are 
provided  below  (see  Appendices  C  and  D  for  the  complete  set  of  statements). 

•  I  found  the  Utilization  Scenario  clear. 

•  I  found  the  P2P  Training  Basics  section  clear. 

•  I  found  the  P2P  Training  Techniques  section  clear. 

Overall  the  clarity  ratings  for  the  educational  and  guideline  materials  were  positive  from 
the  first  and  second  evaluations  with  the  average  rating  across  all  items  being  M=  4.08  (SD  = 
.60),  and  M—  3.94  (SD  =  .56),  respectively;  the  median  response  was  4-Agree  for  both  courses. 
As  shown  in  Figure  7,  none  of  the  respondents  indicated  that  they  disagreed  with  any  of  the 
statements,  while  a  strong  majority  (>  80%)  responded  that  they  Agreed  or  Strongly  Agreed  with 
the  statements. 
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Figure  7.  Response  distributions  for  evaluating  the  clarity  of  the  Instructor’s  Guide  (minus  job  aids). 

After  completing  the  Likert-scale  items,  the  respondents  were  asked  to  provide  written 
comments.  Only  one  instructor  offered  feedback  during  the  first  evaluation  suggesting  that  the 
materials  should  be  broken  into  two  parts,  “a  job  aid  and  a  user  guide.”  During  the  second 
evaluation  one  course  leader  commented,  “All  of  the  parts  are  very  useful.” 

The  guide  contained  15  pages  of  job  aids  designed  to  assist  instructors  in  indentifying 
opportunities  to  implement  P2P  training,  indentify  threats  to  training,  and  assessing  progress. 

The  clarity  of  the  job  aids  were  evaluated  through  eight  written  statements  during  the 
evaluations.  Sample  statements  appear  below  (see  Appendices  C  and  D  for  the  complete  set  of 
statements). 

•  I  found  Recognizing  P2P  Training  Opportunities  clear. 

•  I  found  Using  Empowering  Statements  clear. 

•  I  found  Using  Guiding  Questions  clear. 

Across  the  board,  the  job  aid  clarity  ratings  were  positive  from  the  first  and  second 
evaluations,  with  the  average  rating  across  all  items  being  M=  4.20  (SD  =  .58)  and  M=  4.05  (SD 
=  .75),  respectively.  The  median  response  was  4-Agree  for  both  evaluations.  As  seen  in  Figure 
8,  no  participant  indicated  that  he  disagreed  with  any  of  the  statements,  and  a  vast  majority  (> 

75%)  responded  that  they  Agreed  or  Strongly  Agreed  with  the  statements. 
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Figure  8.  Response  distributions  for  evaluating  clarity  of  job  aids. 

The  suitability  of  the  job  aids  were  evaluated  through  14  written  statements  during  both 
evaluations.  Representative  statements  are  provided  below  (see  Appendices  C  and  D  for  the 
complete  set  of  statements). 

•  The  Assessment  of  Student  job  aids  measure  the  right  skills. 

•  The  Assessment  of  Student  job  aids  measure  the  right  attributes. 

•  The  Assessment  of  Student  job  aids  can  be  used  in  the  classroom. 

•  The  Assessment  of  Instructor  job  aid  can  be  used  at  the  end  of  a  block  of  instruction. 

On  the  whole,  the  job  aid  suitability  ratings  were  positive  during  the  first  and  second 
evaluations,  averaging  3.98  ( SD  =  .94)  and  3.95  ( SD  =  .72),  respectively,  with  a  median  of  4- 
Agree  for  both  courses.  As  Figure  9  shows,  respondents  indicated  that  they  Disagreed  or 
Strongly  Disagreed  with  some  of  the  statements.  However,  it  is  important  to  note  the  items 
receiving  these  scores  were  reverse-scored  items.  That  is,  lower  scores  reflect  a  more  positive 
rating.  The  two  reverse-scored  statements  were  “The  Assessment  of  Student  Job  aids  take  too 
long  to  implement”  and  “The  Assessment  of  Instructor  job  aids  takes  too  long  to  implement.” 
All  of  the  Strongly  Disagree  and  Disagree  responses  were  for  these  two  items. 
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Figure  9.  Response  distributions  for  evaluating  suitability  of  job  aids. 

Once  again,  comments  were  few  but  informative.  Two  comments  were  provided  during 
the  first  evaluation,  “My  disagreement  comes  from  the  observation  that  intangibles  that  the  ARC 
has  outlined  don’t  appear  to  be  addressed.  Apples  and  oranges,”  and  “Make  it  smaller.”  No 
comments  were  provided  during  the  second  evaluation. 

Utility  of  the  Guide 

The  utility  of  the  guide  was  evaluated  across  several  dimensions.  Ratings  and  written 
and  verbal  feedback  were  used  to  assess  the  time  requirements  and  frequency  of  guide  use. 
Ratings  and  feedback  were  also  collected  on  the  perceived  usefulness  of  the  guide.  Finally, 
queries  examined  the  degree  to  which  the  guide  was  perceived  to  meet  the  needs  of  the  course. 

There  was  a  fair  amount  of  instructor  turnover  within  and  between  each  course.  About 
40%  of  the  instructors  were  in  their  first  cycle  or  indicated  that  they  had  been  an  instructor  for 
less  than  six  months.  The  turnover  rate  and  instructor  training  affected  instructors’  use  of  the 
guide.  Following  the  first  course,  approximately  25%  of  the  respondents  indicated  they  were  not 
completely  familiarized  with  the  guide,  while  following  the  second  course,  approximately  36% 
of  the  respondents  indicated  that  they  did  not  familiarize  themselves  with  the  latest  version  of  the 
guide.  The  most  frequent  reason  cited  was  being  new  to  the  course  and  busy  learning  course 
material  or  still  training  to  become  an  instructor. 

The  time  spent  studying  the  guide  was  examined  in  the  second  evaluation.  Instructors 
reported  the  amount  of  time  they  spent  studying  the  guide  averaged  1  hour  and  45  minutes. 
Instructors  indicated  the  amount  of  time  spent  using  the  guide  was  reasonable. 
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Of  the  instructors  who  responded  to  the  question,  the  most  frequently  cited  uses  of  the 
guide  across  both  courses  were  self-study  and  preparation  for  training  (see  Table  1). 

Table  1 

Self-Reported  Usage  of  Instructor’s  Guide,  by  Application 


Application 

Self-Study 

Preparation  for  Training 
Execution  of  Training 
Assessment  of  Students 


First  Evaluation 
68.75% 
50.00% 
25.00% 
43.75% 


Second  Evaluation 
54.54% 
54.54% 
36.36% 
27.27% 


Although  differences  emerged  in  the  reported  frequency  of  use  according  to  application, 
comments  from  those  instructors  who  used  it  across  all  applications  indicated  its  usefulness  in  all 
areas.  For  example,  “Everyone  needs  to  use  this  and  not  just  for  the  designated  leader.  But  for 
all  the  students  in  the  course  -  all  the  time.”  Comments  from  course  leaders  also  indicated  its 
usefulness,  as  the  following  illustrates: 

“I  used  it  to  guide  my  assessment  of  instructors.  From  Course  Leader’s  position,  this 
guide  keeps  me  centered  on  course  outcomes.  I  am  able  to  judge  and  assess  instructors  as 
they  teach  students.  Holding  this  guide  in  my  hand,  I  can  observe  and  assess  the 
instructor  in  terms  of  outcomes  and  P2P  learning.  This  guide  nudges  and  pushes  me  to 
ensure  instructors  are  teaching  IAW  POI  [in  accordance  with  the  program  of  instruction] 
to  grow  adaptable,  flexible  leaders.” 

The  usefulness  of  the  guide’s  educational  and  guideline  materials  was  evaluated  through 
eight  written  statements  (first  evaluation)  and  nine  written  statements  (second  evaluation)  which 
examined  various  aspects  of  the  materials.  Sample  statements  appear  below  (see  Appendices  C 
and  D  for  the  complete  set  of  statements). 

•  I  found  the  Utilization  Scenario  useful. 

•  I  found  the  P2P  Training  Basics  section  useful. 

•  I  found  the  P2P  Training  Techniques  section  useful. 

Overall  the  usefulness  ratings  for  the  educational  and  guideline  materials  were  positive  in 
both  the  first  and  second  evaluations,  the  average  rating  across  all  items  being  M=  4.02  (SD  = 
.64),  and  M=  3.94  (SD  =  .53),  respectively;  median  response  was  4-Agree  for  both  evaluations. 
As  Figure  10  shows,  no  participants  indicated  that  they  disagreed  with  any  of  the  statements, 
while  a  strong  majority  (>  80%)  Agreed  or  Strongly  Agreed  with  the  statements. 
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Figure  10.  Response  distributions  for  evaluating  usefulness  of  Instructor’s  Guide  (minus  job  aids). 

The  usefulness  of  the  job  aids  was  evaluated  through  eight  written  statements  during  both 
evaluations.  Sample  statements  are  provided  below  (see  Appendices  C  and  D  for  the  complete 
set  of  statements). 

•  I  found  Recognizing  P2P  Training  Opportunities  useful. 

•  I  found  Exploiting  P2P  Potential  useful. 

•  I  found  Using  Guiding  Questions  useful. 

Taken  as  a  whole,  the  job  aid  usefulness  ratings  were  positive  in  both  evaluations,  with 
the  average  rating  across  all  items  being  M  =  4.16  ( SD  =  .65)  and  M  =  4.07  ( SD  =  .75), 
respectively.  The  median  response  was  4-Agree  for  both  evaluations.  As  seen  in  Figure  1 1,  no 
respondents  indicated  that  they  disagreed  with  any  of  the  statements,  and  the  majority  (>75%) 
responded  that  they  Agreed  or  Strongly  Agreed  with  the  statements.  Only  one  comment  was 
provided  during  each  evaluation  regarding  the  usefulness  of  the  job  aids.  Both  comments  were 
very  positive;  one  of  the  course  leaders  stated,  “I  love  the  job  aids,”  and  “This  is  my  favorite 
part.” 
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Figure  11.  Response  distributions  for  evaluating  usefulness  of  job  aids. 

Ease  of  use  was  explored  by  soliciting  participant  comments  concerning  problems  they 
encountered  in  understanding  or  using  the  guide.  Very  few  problems  were  reported.  During  the 
first  evaluation  a  couple  of  instructors  commented  that  the  only  problem  they  had  was  finding 
the  time/opportunities  to  employ  it.  One  other  instructor  commented  that  he  found  the  8.5  by  1 1 
inch  size  (landscape  layout)  difficult  to  use  in  the  field  and  suggested  the  research  team  develop 
a  version  of  the  guide  printable  on  smaller  paper  to  allow  instructors  to  carry  it  in  their  pockets. 
This  led  to  production  of  a  4.5  by  5.7  inch  version  (landscape  layout)  for  the  second  evaluation. 
During  the  second  evaluation,  one  instructor  commented  that  he  forgot  to  use  the  guide  several 
times  when  he  could/should  have,  and  one  commented  that  the  guide  was  too  small. 

Generally,  the  comments  conveyed  the  impression  that  the  materials  fulfilled  the  needs  of 
the  course.  Comments  from  the  first  evaluation  indicated  that  instructors  and  course  leaders 
found  the  materials  provided  good  materials  for  understanding  OBT&E,  guidance  for  instructors, 
and  assessment  tools  for  students  and  instructors.  Comments  from  the  second  evaluation  echoed 
the  positive  responses  received  during  the  first  evaluation.  Some  representative  comments: 

•  “Easy  to  reference  before,  during,  or  after  training  to  focus  your  effort.” 

•  “As  a  new  hire  just  beginning  to  familiarize  myself  with  the  OBT&E  process,  the 
guide  offers  a  breakdown  and  examples  of  the  process  for  easy  understanding  and 
implementation.” 


24 


•  “I  am  new  to  OBT&E,  but  the  guide  helped  me  substantially  because  it  helped  me  ask 
the  right  questions  and  allowed  me  to  steer  students  in  the  right  direction.” 

•  “It  makes  for  highly  effective,  professional  instruction.” 

Suggested  Improvements  to  the  Guide 

The  participants  offered  occasional  ideas  for  improving  the  guide.  The  areas  identified 
for  improvement  during  the  first  evaluation  were  the  format  and  the  relative  fit  with  OBT&E 
principles.  In  the  same  evaluation  the  most  commonly  cited  obstacle  to  using  the  guide  was  its 
size  (height  and  width).  Several  instructors  stated  they  would  like  a  pocket-sized  guide  to  use  in 
the  field.  Based  upon  this  feedback  a  pocket-sized  version  was  produced  for  the  second  course. 
While  receiving  positive  feedback  from  most  instructors,  some  requested  that  the  pocket-sized 
guide  be  slightly  larger,  while  others  suggested  that  it  should  be  slightly  smaller.  A  waterproof/ 
laminated  version  was  also  requested,  because  “instructors  spend  extensive  time  in  the  field” 
where  conditions  are  rugged. 

One  of  the  instructors  noted  during  the  first  evaluation  that  the  ARC  intangibles  “. .  .don’t 
appear  to  be  addressed”  in  the  guide.  The  first  version  of  the  guide  had  included  the  initial  set  of 
intangibles  developed  for  the  ARC’S  transition  to  the  OBT&E  method.  However,  these  had  been 
refined  by  the  ARC  cadre  between  the  guide  development  phase  and  the  first  evaluation. 
Subsequently,  the  research  team  revised  the  guide  to  accurately  reflect  the  most  recent  version  of 
the  ARC  intangibles. 

During  the  first  evaluation,  one  instructor  commented  “there  needs  to  be  closer  match  to 
OBT&E  to  assess  intangibles  of  ARC.”  Based  upon  this  comment  the  research  team  reviewed 
the  materials  pertaining  to  the  intangibles  and  modified  them  to  better  mirror  the  course 
objectives.  The  remaining  feedback  was  very  positive.  One  course  leader  stated,  “The  only 
thing  to  improve  on  the  guide  is  remove  the  word  ‘Draft’  from  the  title.”  During  the  second 
evaluation,  no  suggestions  for  improvement  were  provided,  indicating  that  both  instructors  and 
course  leaders  were  sufficiently  satisfied  with  the  maturity  of  the  guide  in  its  current  fonnat. 

Lessons  Learned 

The  ARI  team,  which  included  ARC  collaborators,  created  instructional  methods  and 
measurement  tools  that  were  well-received  by  users,  assisted  in  training,  and  supported  the 
assessment  of  intangible  attributes.  The  active  collaboration  between  the  ARC  cadre  and  ARI 
researchers  was  most  likely  a  key  factor  in  the  successful  development.  Equally  important,  it 
seems,  was  the  careful  alignment  of  the  methods  and  tools  with  the  instructional  demands  placed 
on  the  cadre.  Another  key  factor  was  the  design  approach  that  emphasized  Soldier-friendly, 
easy-to-use  features.  “Lean  and  simple”  are  important  characteristics  when  implementation 
takes  place  in  a  working  environment  with  severe  time  pressures.  The  data  do  not  indicate 
whether  the  declarative,  self-development  materials  were  more  or  less  valuable  than  the  job  aids, 
but  the  inclusion  of  practical  tools  for  accomplishing  job  duties  is  considered  imperative. 

Several  limitations  in  the  formative  evaluation  techniques  surfaced.  Ratings  and 
comments  did  not  change  much  across  revisions  of  the  guide,  likely  because  instructors  were 
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focusing  on  their  primary  duties.  In  addition,  the  course  leaders  may  have  felt  that  the  feedback 
queries  were  lengthy  and  redundant.  Perhaps  simpler,  global  queries  would  be  preferable,  such 
as  “Is  this  guide  ready  for  adoption/implementation?”  Participants’  level  of  interest  and  effort  in 
furnishing  feedback  might  be  increased  by  using  probes  to  evaluate  various  aspects  of  the 
instructional  and  measurement  methods  across  revisions.  Unfortunately,  both  approaches  would 
also  limit  the  usefulness  of  the  resulting  data. 

Conclusions 

Soldiers’  endorsement  of  the  Instructor’s  P2P  Learning  Guide  for  the  ARC  indicates  that 
the  team  of  ARI  researchers  and  ARC  cadre  members  successfully  constructed  instructional 
methods  and  measurement  tools  to  support  FSTD  in  the  ARC  program.  Using  P2P  and  OBT&E 
principles  as  a  foundation,  the  process  outlined  in  this  report  could  be  used  to  create  instructional 
methods  and  measurement  techniques  for  a  variety  of  other  courses. 

Many  participants  commented  that  the  guide  is  a  good  tool,  resource,  and  reference  for 
instructors  to  glean  questions,  scenarios,  and  techniques  for  use  during  training.  Feedback  also 
suggested  the  guide  is  easy  to  understand  and  helps  instructors  familiarize  themselves  with  the 
P2P  training  method.  Instructors  found  the  guide  helpful  in  executing  their  instructional  duties. 
Overall,  the  comments  on  the  assessment  section  of  the  guide  were  positive.  The  assessment 
tools  were  perceived  as  a  valuable  resource  in  both  the  field  and  the  classroom.  Following  the 
second  evaluation,  one  course  leader  commented,  “Will  include  in  the  ARC  course  as  standard 
operating  procedure.” 

During  operational  implementation,  the  primary  aspect  of  the  guide  that  was  identified 
for  improvement  was  physical  size  (height  and  width).  Instructors  requested  a  smaller  version  of 
the  guide  that  could  easily  fit  into  an  Army  Combat  Uniform  (ACU)  pocket.  A  pocket  version  of 
the  guide  was  produced  for  the  second  evaluation  and  drew  positive  feedback.  Based  on  the 
second  evaluation,  respondents  found  little  else  to  improve  in  the  guide,  including  job  aids. 
However,  future  modifications  may  emerge  as  instructors  become  more  seasoned  in 
implementing  P2P  training  during  ARC  cycles. 

The  guide  provides  a  valuable  tool  to  foster  sound  instructorship.  It  provides  simple, 
straightforward  methods  that  can  be  adapted  for  a  variety  of  different  learning  environments  - 
classroom,  counseling  sessions,  field  exercises  -  using  P2P  and  OBT&E  as  a  general  platform. 
The  methodology  outlined  in  this  report  offers  a  roadmap  for  developing  materials  to  be  used 
throughout  all  phases  of  a  training  course.  The  principles  outlined  follow  a  crawl,  walk,  run 
model.  They  provide  workable  measures  of  students’  proficiency  level,  identify  areas  that  need 
improvement,  and  determine  when  course  goals  have  been  reached.  The  formative  evaluation 
findings  suggest  the  research  team  and  ARC  cadre  created  an  acceptable,  suitable,  usable,  and 
valuable  Instructor’s  Guide. 


Future  Directions 

The  Instructor’s  P2P  Learning  Guide  for  the  ARC  has  become  a  tool  for  professional 
development  of  the  ARC  cadre.  Dissemination  and  implementation  of  the  guide  have  already 
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occurred.  The  pocket-sized  version  was  finalized  based  on  cadre  feedback  and  posted  on  the 
ARC  server.  Weather-proof,  laminated  copies  of  the  guide  were  provided  to  enhance  the 
portability  of  the  P2P  training  materials.  The  guide  has  been  very  useful  to  the  cadre  as  they 
conduct  P2P  training,  and  its  continued  utilization  seems  assured.  Over  time,  the  guide  will 
require  maintenance  and  updating  to  ensure  it  stays  accessible  and  current. 

The  guide  was  posted  on  the  Army  Training  Network  Web  site  to  facilitate  introduction 
and  dissemination  among  Army  stakeholders.  The  Web  site  is  dedicated  to  making  Army 
training  management  products  accessible.  The  following  short  synopsis  of  the  guide’s  purpose 
accompanies  the  website  posting:  “The  Instructor's  P2P  Learning  Guide  is  a  tool  to  assist  ARC 
instructors  in  developing  reconnaissance  leader  skills  and  attributes  in  an  OBT&E  learning 
environment.  This  guide  provides  ARC  instructors  guidelines  for  implementing  and  assessing 
P2P  learning  as  well  as  job  aids  to  train  and  assess  recon  leader  skills  and  attributes.” 

With  the  ARC’S  expanded  responsibilities  to  train  reconnaissance  leaders  for  the  entire 
Army,  the  implications  for  implementing  the  guide  are  long-tenn.  The  transition  of  the  ARC 
program  aligns  with  the  Army  Leader  Development  Strategy  (Department  of  the  Army,  2009c). 
The  strategy  requires  Army  leaders  to  (a)  confront  complex,  dynamic,  and  unanticipated 
challenges,  (b)  be  competent  in  core  proficiencies  and  broad  enough  to  operate  across  the 
spectrum  of  conflict,  and  (c)  enable  innovative  and  adaptive  leaders  at  the  lowest  levels.  As 
implementation  of  the  leader  development  strategy  moves  forward,  the  role  of  P2P  training  is 
likely  to  increase  across  the  force.  The  Instructor’s  P2P  Learning  Guide  for  the  ARC  embodies 
learning  principles  that  are  applicable  to  other  Army  courses  and  leader  training  functions. 
Additional  work  could  be  conducted  to  tailor  instructor’s  guides  to  specific  needs  of  other  Army 
units  and  courses. 

The  beneficial  aspects  of  P2P  training  principles  that  enable  FSTD  extend  beyond  the 
ARC  program.  Instructional  methods  and  measurement  techniques  can  be  designed  for  other 
courses  based  on  the  methodology  described  in  this  report.  Guides  can  be  tailored  to  the  specific 
skills  and  attributes  of  a  particular  specialty  group  to  support  FSTD.  Principles  of  P2P  training 
are  applicable  to  a  wide  range  of  training  and  education  programs  and  fit  well  with  traditional 
methods  of  instruction.  The  process  outlined  in  this  report  serves  as  a  “proof  of  concept”  for  a 
method  to  develop  instructor’s  tools  to  support  FSTD. 

Soldiers’  experience  levels  should  be  considered  when  determining  how  to  incorporate 
P2P  principles  into  training.  For  example,  the  type  and  extent  of  P2P  training  in  basic  training 
versus  a  leader’s  course  may  differ.  Soldiers  who  are  experts  in  one  area  may  benefit  greatly 
from  sharing  their  knowledge  and  experiences  with  experts  in  another  area,  and  therefore  P2P 
principles  could  be  incorporated  more  extensively  into  training.  Conversely,  Soldiers  with  little 
or  no  experience  may  benefit  from  greater  levels  of  guidance  from  instructors  than  inexperienced 
peers.  Course  developers  need  to  be  mindful  of  such  concerns  when  developing  instructional 
methods  and  measurement  techniques. 

The  need  to  implement  P2P  training  principles  is  underscored  by  the  constantly  changing 
conditions  of  FSO.  The  current  military  climate  cycles  Soldiers  through  multiple  deployments. 
Each  deployment  brings  new  challenges  which  require  the  ability  to  adapt  quickly  to  a  variety  of 
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situations.  Adversaries  are  constantly  modifying  their  strategies  and  techniques.  Civilian  factors 
vary  from  village  to  village  and  region  to  region.  The  complexity  of  FSO  requires  Soldiers  to 
maintain  a  broad  and  deep  base  of  knowledge.  Soldiers  can  benefit  greatly  from  sharing  the 
knowledge  and  lessons  learned  they  acquire  during  deployment.  The  P2P  training  methodology 
presented  in  this  report  can  create  a  natural  forum  for  sharing  hard-won  information.  The  guide 
can  help  Soldiers  and  instructors  focus  on  “how”  and  “why”  as  well  as  the  skills  and  attributes 
required  to  successfully  adapt  to  the  inherent  uncertainties  and  complexities  of  military 
operations. 

In  summary,  the  research  reported  here  was  conducted  to  help  prepare  Soldiers  address 
the  challenges  of  FSO  and  particularly  COIN.  A  striking  example  of  today’s  operational 
challenges  is  exemplified  by  the  role  of  reconnaissance  leaders.  In  response,  the  Anny  and 
TRADOC  now  stress  the  need  to  realize  the  full  spectrum  of  human  capabilities  required  for 
FSO.  The  Instructor’s  Guide  developed  for  ARC  can  be  readily  adapted  to  foster  the  skills  and 
attributes  required  by  other  Soldiers  and  leaders. 
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Appendix  A 

Acronyms  and  Abbreviations 


AAR 

ACU 

ARC 

ARI 

AWG 

BCT 

CATC 

CCIR 

Cdr 

COIN 

FM 

FSO 

FSTD 

IAW 

ID 

IPB 

M 

MOA 

NCO 

OBT&E 

OIC 

P2P 

POI 

RL 

SD 

STP 

TRADOC 


After  Action  Review 
Army  Combat  Uniform 
Army  Reconnaissance  Course 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
Asymmetric  Warfare  Group 

Brigade  Combat  Team 

Combat  Applications  Training  Course 
Commander’s  Critical  Information  Requirements 
Commander 
C  ounterinsurgency 

Field  Manual 

Full  Spectrum  Operations 

Full  Spectrum  Training  and  Development 

In  Accordance  With 
Identification 

Intelligence  Preparation  of  the  Battlefield 
Mean 

Minutes  of  Angle 

Noncommissioned  Officer 

Outcomes-Based  Training  and  Education 
Officer  in  Charge 

Peer-to-Peer 
Program  of  Instruction 

Reconnaissance  Leader 

Standard  Deviation 
Soldier  Training  Publication 

U.S.  Army  Training  and  Doctrine  Command 
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Appendix  B 


OBSERVATION  GUIDE 


GOAL 

The  goal  of  observing  ARC  classes  and  exercises  is  to  document  OBT&E 
and  P2P  training  techniques  and  tools  used  by  instructors  and  students. 


Observer  Duties: 

1 .  Capture  notes  on  questions  of  interest  plus  administrative  data 

2.  Organize  and  share  notes  and  insights  from  your  observations 

Instructions: 

•  Study  available  instructional  materials  (e.g.,  lesson  plans)  in  advance  of  the  event. 

•  Become  familiar  with  the  questions  of  interest  listed  on  page  2. 

•  Touch  base  with  the  instructor  or  officer  in  charge  (OIC)  at  the  start  of  the  session. 

•  Assure  the  instructor  we  are  not  evaluating  him,  the  students,  or  the  POI. 

•  Take  notes  while  you  observe,  using  the  questions  of  interest  as  a  guide. 

•  Focus  on  what  behavioral  examples  are  observed. 

•  Avoid  influencing,  assisting,  or  interfering  with  the  conduct  of  the  activities. 

•  Within  3  days,  put  your  notes  into  a  Microsoft  Word  file  and  share  with  the  team. 


ADMINISTRATIVE  DATA 

A.  Observer’s  Name _ 

B.  Date  of  Observation _ 

C.  Name  of  Class  or  Exercise _ 

D.  #  Instructors  and  Their  Roles _ 

E.  #  Students  and  Their  Grouping _ 

F.  Brief  Description  of  Activity _ 


G.  Observation  Start  Time 

H.  Observation  End  Time 
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QUESTIONS  OF  INTEREST 


How  are  intangibles  taught? 

1 .  Which  intangible  attributes  do  you  see  the  instructor  address  during  the  session? 

Confidence  Accountability  Initiative  Anticipation  Problem-Solving  Judgment  Decision-Making 

2.  How  does  the  instructor  direct  the  students’  attention  to  the  intangibles  (e.g.,  link  training  to 
course  outcomes)? 

3.  How  does  instructor  model  intangibles  (e.g.,  Socratic  questioning,  empowering  statements)? 

4.  What  %  of  time  does  the  instructor  spend  teaching  or  addressing  intangibles? 

5.  Which  methods  of  instruction  are  used:  (a)  lecturing,  (b)  discussing,  (c)  demonstrating, 

(d)  practicing,  (e)  coaching/mentoring/advising,  (f)  reviewing,  (g)  testing,  and  (h)  other? 

What  OBT&E  features  are  applied? 

6.  What  are  the  problem  sets  for  the  training  session? 

7.  How  are  the  problem  sets  linked  to  the  course  outcomes? 

8.  What  steps  are  used  during  training  to  enhance  student  awareness  of  the  link  between  the 
problem  sets  and  the  course  outcomes? 

9.  How  does  the  instructor  apply  OBT&E  methods?  Describe  the  activities  and  techniques. 

What  P2P  training  principles  are  employed? 

10.  How  do  the  students  interact  with  the  instructor(s)?  Describe  what  you  see. 

11.  How  do  the  students  interact  with  each  other?  Describe  what  you  see.  (Note:  Look  for 
factors  that  facilitate  or  inhibit  sharing/accepting  knowledge  and  strengthening  intangibles.) 

12.  When  do  you  see  students  forming  splinter  groups?  Why  do  they  work  separately? 

How  do  feedback  sessions  address  OBT&E,  especially  intangible  attributes? 

13.  Describe  feedback  sessions  for  individuals  and  groups. 

14.  What  topics  or  issues  are  addressed  during  feedback  sessions? 

15.  How  is  the  feedback  linked  to  course  outcomes? 

What  do  instructor  huddles  address? 

16.  What  OBT&E  topics,  particularly  intangibles,  are  discussed  during  instructor  huddles? 

17.  Are  P2P  training  techniques  used  to  improve  OBT&E  processes?  Describe  what  you  see. 

18.  Do  the  instructors  share  lessons  learned  about  OBT&E  methods?  Describe  how. 

Other  Questions 

19.  What  happens  during  self-assessment  and  peer-assessment  events? 

20.  When  and  how  could  measures  of  instructor  effectiveness  be  obtained? 

21 .  When  and  how  could  measures  of  group  dynamics  be  obtained? 

22.  What  lessons  do  you  see  for  incorporating  P2P  training  methods  into  ARC  activities? 
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OBT&E  PROCESS  SUMMARY 

•  Course  outcomes  are  identified  and  used  to  guide  training. 

•  Problem  sets  are  introduced  to  facilitate  progress  toward  achieving  course  outcomes. 

•  Facilitators  use  Socratic  questioning  to  guide  students  to  accept  responsibility  for  achieving, 
self  development,  training  success  and  course  outcomes. 

OBT&E  RESULTS 

•  Soldiers,  leaders,  and  units  who  teach  themselves. 

•  Soldiers  who  solve  problems  as  individuals  and  teams  based  on  knowledge  and  principles. 

•  Empowerment  by  mastery  of  tangible  skills  and  possession  of  intangible  attributes. 

ARC  INTENDED  OUTCOMES 

•  Higher  fundamental  recon  skills  -  land  navigation,  communication/reporting,  tactical  analysis. 

•  Better  understanding  of  higher  commander’s  (Cdr)  info  requirements,  how  to  find  and 
communicate  info. 

•  Higher  skills  at  planning  and  executing  w/out  compromising  mission  or  freedom  of  action. 

•  Competence  with  employment  of  organic  and  attached  assets  -  air,  ground,  technical. 

•  Confidence  in  mission-relevant  judgment,  problem-solving,  anticipation,  initiative,  risk 
management,  accountability. 

INTANGIBLE  ATTRIBUTES 

•  Confidence  -  belief  in  own  ability  to  handle  tactical  situations. 

•  Accountability  -  willingness  to  take  responsibility  for  own  actions  and  consequences. 

•  Initiative  -  tendency  to  think  and  act  without  being  urged. 

•  Anticipation  -  ability  to  think  ahead  and  project  future  requirements  or  conditions. 

•  Problem-solving  -  inclination  to  approach  problems  by  applying  deliberate  thought. 

•  Judgment  -  ability  to  reach  sensible  conclusions  in  light  of  available  information. 

•  Decision  making  -  propensity  to  make  decisions  in  timely  manner. 


P2P  TRAINING  BEST  PRACTICES  AND  OBT&E  APPLICATION 


P2P  TRAINING 

OBT&E 

Assign  group  tasks  to  generate  active 
involvement  and  ideas 

Identify  problem  sets 

Establish  measurable  training  objectives 

Link  training  to  the  problem  sets/course 
outcomes 

Avoid  traditional  “instructor-student”  roles 

Students  assume  responsibility  for  learning 

Ask  open-ended  questions  to  guide  dialogue 

Use  Socratic  questioning  and  empowering 
statements  to  guide  students 

Incorporate  activities  to  promote  social 
interaction 

Configure  problem-focused  teams,  conduct 
feedback  sessions 

Use  different  group  configurations  to  generate 
information  exchanges 

Instructor  adjusts  learning  demands  to  maximize 
student  interaction  or  collaboration 

Require  Soldiers  to  apply  new  knowledge  during 
the  session 

Demonstrate  intangibles  by  applying  what’s 
learned 

Leverage  instructor  as  a  model  of  expert  learning  process 

B-3 


Project  Facts 

Title:  Method  and  Measure  Refinements  for  Outcomes  Based  Training  (OBT&E) 

Performing  organization:  U.S.  Army  Research  Institute  -  Fort  Knox 
Period  of  performance:  16  Mar  09  to  15  Mar  10  (12  months) 

Project  goal:  Use  peer-to-peer  (P2P)  training  methods  to  facilitate  Full  Spectrum  Training  in 
ARC 

Products:  P2P-based  training  methods  and  assessment  measures  for  use  by  ARC  instructors 


Project  Method  and  Outcomes 

•  Observe  and  support  ongoing  efforts  to  apply  the  OBT&E  approach  to  ARC. 

•  Develop  P2P-type  training  methods  to  improve  the  ability  of  ARC  instructors  to  model 
intangible  Soldier  attributes.  This  includes  developing  guidance  and  job  aids  for  ARC 
instructors  based  on  a  previously  developed  P2P  training  facilitator’s  guide. 

•  Develop  P2P-type  measurement  methods  to  assess  how  well  ARC  instructors  model 
intangible  Soldier  attributes. 

•  Package  the  P2P-type  training  and  assessment  methods  in  an  instructor-friendly  guide. 

•  Implement  the  training  and  measurement  methods,  and  obtain  feedback  from  ARC  course 
developers,  instructors,  and  students. 

•  The  P2P-Focused  Instructor’s  Guide  may  become  a  tool  for  professional  development  of 
ARC  cadre. 
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Appendix  C 

ARC  INSTRUCTOR  FEEDBACK  QUESTIONNAIRE 


Date : 


Instructions:  The  questions  below  ask  for  your  opinions  about  the  Instructor’s  P2P  Training 
Guide  provided  by  ARI.  While  answering  these  questions,  please  focus  on  your  reactions  and 
experiences  throughout  the  course.  Write-in  comments,  both  positive  and  negative,  are 
encouraged.  Please  use  a  separate  sheet  of  paper  if  you  need  additional  space. 


1.  How  often  did  you  study  or  apply  the  various  parts 
of  the  P2P  Training  Guide? 

Circle  One  for  Each  Item 

Never 

Once 

Twice 

Three 

Times 

Four/+ 

Times 

a.  Utilization  Scenario 

1 

2 

3 

4 

5 

b.  Training  Basics 

1 

2 

3 

4 

5 

c.  Learning  Model  (diagram) 

1 

2 

3 

4 

5 

d.  Best  Practices  for  P2P  Training 

1 

2 

3 

4 

5 

e.  Setting  P2P  Training  Conditions 

1 

2 

3 

4 

5 

f.  Recon  Leader  Skills  and  Attributes 

1 

2 

3 

4 

5 

g.  Training  Techniques  and  Action  Guidelines 

1 

2 

3 

4 

5 

h.  Best  Practices  for  P2P  Training  Assessment 

1 

2 

3 

4 

5 

i.  Assessment  Techniques  and  Action  Guidelines 

1 

2 

3 

4 

5 

j.  Job  Aid:  Recognizing  P2P  Training  Opportunities 

1 

2 

3 

4 

5 

k.  Job  Aid:  Using  Guiding  Questions 

1 

2 

3 

4 

5 

1.  Job  Aid:  Exploiting  P2P  Training  Potential 

1 

2 

3 

4 

5 

m.  Job  Aid:  Using  Empowering  Statements 

1 

2 

3 

4 

5 

n.  Job  Aid:  Deterring  Threats  to  Training 

1 

2 

3 

4 

5 

o.  Job  Aid:  Student  Assessments 

1 

2 

3 

4 

5 

p.  Job  Aid:  Instructor  Assessments 

1 

2 

3 

4 

5 

q.  Job  Aid:  Hotwash/After  Action  Review  (AAR) 

1 

2 

3 

4 

5 

Comments  and  Suggestions: 

2.  When  did  you  use  the  guide?  (Circle  all  that  apply) 

Self-Study  Preparation  for  Training  Execution  of  Training 

Assessment  of  Students 

Please  explain : 
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3.  What  problems  did  you  encounter  in  understanding  or  using  the  guide? 


4.  What  was  the  most  useful  part  of  the  guide?  (Circle  one) 

Utilization  Scenario  P2P  Training  Basics  &  Process  P2P  Training  Guidelines/Techniques 

P2P  Training  Assessment  Guidelines/Techniques  P2P  Training  Job  Aids  P2P  Training 

Assessment  Job  Aids 

Hotwash/AAR  Job  Aid 
Please  explain  why. 


5.  What  could  be  added  to  the  guide  to  better  assist  you  as  an  instructor? 


6.  What  could  be  eliminated  from  the  guide  without  degrading  its  usefulness? 
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7.  How  do  you  feel  about  the  clarity  and  usefulness  of  the 
sections  of  the  quide’s  instructional  materials? 

Circle  One  for  Each  Item 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

a.  I  found  the  Utilization  Scenario  clear 

1 

2 

3 

4 

5 

b.  I  found  the  Utilization  Scenario  useful 

1 

2 

3 

4 

5 

c.  I  found  the  P2P  Training  Basics  section  clear 

1 

2 

3 

4 

5 

d.  I  found  the  P2P  Training  Basics  section  useful 

1 

2 

3 

4 

5 

e.  I  found  the  ARC  P2P  Training  Model  clear 

1 

2 

3 

4 

5 

f.  I  found  the  ARC  P2P  Training  Model  useful 

1 

2 

3 

4 

5 

g.  I  found  the  P2P  Training  Best  Practices  section  clear 

1 

2 

3 

4 

5 

h.  I  found  the  P2P  Training  Best  Practices  section  useful 

1 

2 

3 

4 

5 

i.  I  found  the  Setting  P2P  Training  Conditions  section  clear 

1 

2 

3 

4 

5 

j.  I  found  the  Setting  P2P  Training  Conditions  section  useful 

1 

2 

3 

4 

5 

k.  I  found  the  Recon  Leader  Skills  and  Attributes  section  clear 

1 

2 

3 

4 

5 

1.  I  found  the  Recon  Leader  Skills  and  Attributes  section  useful 

1 

2 

3 

4 

5 

m.  I  found  the  P2P  Training  Techniques  section  clear 

1 

2 

3 

4 

5 

n.  I  found  the  P2P  Training  Techniques  section  useful 

1 

2 

3 

4 

5 

o.  I  found  the  P2P  Training  Assessment  Best  Practices  section  clear 

1 

2 

3 

4 

5 

p.  I  found  the  P2P  Training  Assessment  Best  Practices  section  useful 

1 

2 

3 

4 

5 

q.  I  found  the  P2P  Training  Assessment  Techniques  section  clear 

1 

2 

3 

4 

5 

r.  I  found  the  P2P  Training  Assessment  Techniques  section  useful 

1 

2 

3 

4 

5 

Comments  and  Suggestions: 

8.  How  do  you  feel  about  the  clarity  and  usefulness  of  the 
job  aids  in  the  quide? 

Circle  One  for  Each  Item 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

a.  I  found  Recognizing  P2P  Training  Opportunities  clear 

1 

2 

3 

4 

5 

b.  I  found  Recognizing  P2P  Training  Opportunities  useful 

1 

2 

3 

4 

5 

c.  I  found  Using  Guiding  Questions  clear 

1 

2 

3 

4 

5 

d.  I  found  Using  Guiding  Questions  useful 

1 

2 

3 

4 

5 

e.  I  found  Exploiting  P2P  Training  Potential  clear 

1 

2 

3 

4 

5 

f.  I  found  Exploiting  P2P  Training  Potential  useful 

1 

2 

3 

4 

5 

g.  I  found  Using  Empowering  Statements  clear 

1 

2 

3 

4 

5 

h.  I  found  Using  Empowering  Statements  useful 

1 

2 

3 

4 

5 

i.  I  found  Deterring  Threats  to  Training  clear 

1 

2 

3 

4 

5 

j.  I  found  Deterring  Threats  to  Training  useful 

1 

2 

3 

4 

5 

k.  I  found  Assessment  of  Student  clear 

1 

2 

3 

4 

5 

1.  I  found  Assessment  of  Student  useful 

1 

2 

3 

4 

5 

m.  I  found  Assessment  of  Instructor  clear 

1 

2 

3 

4 

5 

n.  I  found  Assessment  of  Instructor  useful 

1 

2 

3 

4 

5 

o.  I  found  the  Hotwash/AAR  Job  Aid  clear 

1 

2 

3 

4 

5 

p.  I  found  the  Hotwash/AAR  Job  Aid  useful 

1 

2 

3 

4 

5 

Comments  and  Suggestions: 
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9.  How  do  vou  feel  about  the  suitability  of  the  assessment 
job  aids  found  in  the  quide? 

Circle  One  for  Each  Item 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

a.  The  Assessment  of  Student  job  aids  measure  the  right  skills 

1 

2 

3 

4 

5 

b.  The  Assessment  of  Student  job  aids  measure  the  right  attributes 

1 

2 

3 

4 

5 

c.  The  Assessment  of  Student  job  aids  take  too  long  to  implement 

1 

2 

3 

4 

5 

d.  The  Assessment  of  Student  job  aids  can  be  used  in  the  classroom 

1 

2 

3 

4 

5 

e.  The  Assessment  of  Student  job  aids  can  be  used  in  field  exercises 

1 

2 

3 

4 

5 

f.  The  Assessment  of  Student  job  aids  can  be  used  for  counseling 

1 

2 

3 

4 

5 

g.  The  Assessment  of  Student  job  aids  can  be  used  at  the  end  of  a  block 

1 

2 

3 

4 

5 

h.  The  Assessment  of  Student  job  aids  can  be  used  at  the  course’s  end 

1 

2 

3 

4 

5 

i.  The  Assessment  of  Student  job  aids  work  well  in  5x8  format 

1 

2 

3 

4 

5 

j.  The  Assessment  of  Instructor  job  aid  measures  the  right  attributes 

1 

2 

3 

4 

5 

k.  The  Assessment  of  Instructor  job  aid  takes  too  long  to  implement 

1 

2 

3 

4 

5 

1.  The  Assessment  of  Instructor  job  aid  can  be  used  at  end  of  a  block 

1 

2 

3 

4 

5 

m.  The  Assessment  of  Instructor  job  aid  can  be  used  at  the  course’s  end 

1 

2 

3 

4 

5 

n.  The  Assessment  of  Instructor  job  aid  works  well  in  5x8  format 

1 

2 

3 

4 

5 

Comments  and  Suggestions: 


14.  How  would  you  improve  the  following  parts  of  the  guide? 

Utilization  Scenario: 

P2P  Training  Basics: 

P2P  Training  Model  (diagram): 

P2P  Training  Guidelines/Techniques: 

Recon  Leader  Skills  and  Attributes: 

P2P  Training  Assessment  Guidelines/Techniques: 

Job  Aids  for  Training: 

Job  Aids  for  Assessment: 

Hotwash/AAR  Job  Aid: 

15.  What  other  comments  or  suggestions  do  you  have? 


Thank  you  for  your  feedback! 
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Appendix  D 


ARC  LEADER  FEEDBACK  QUESTIONNAIRE 


Date : 


Instructions:  The  questions  below  ask  for  your  opinions  about  the  Instructor’s  P2P  Training 
Guide  provided  by  ARI.  While  answering  these  questions,  please  focus  on  your  reactions  and 
insights  while  the  guide  was  being  used.  Write-in  comments,  both  positive  and  negative,  are 
encouraged.  Please  use  a  separate  sheet  of  paper  if  you  need  additional  space. 


3.  What  problems  did  you  encounter  in  understanding  or  using  the  guide? 
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4.  How  much  do  you  agree  or  disagree  with  the  following 
statements  about  the  guide? 

Circle  One  for  Each  Item 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

a.  The  guide  provides  information  that  is  relevant  to  the  ARC  POI 

1 

2 

3 

4 

5 

b.  The  guide  supports  the  various  ARC  instructional  activities 

1 

2 

3 

4 

5 

c.  The  guide  can  help  the  cadre  sharpen  their  instructional  practices 

1 

2 

3 

4 

5 

d.  The  guide  can  help  improve  the  student  learning  process 

1 

2 

3 

4 

5 

e.  The  guide  can  help  instructors  achieve  desired  course  outcomes 

1 

2 

3 

4 

5 

f.  The  guide  explains  P2P  training  principles  and  practices  adequately 

1 

2 

3 

4 

5 

g.  The  guide  identifies  the  appropriate  Recon  Leader  Skills 

1 

2 

3 

4 

5 

h.  The  guide  properly  defines  each  Recon  Leader  Skill 

1 

2 

3 

4 

5 

i.  The  guide  identifies  the  appropriate  Recon  Leader  Attributes 

1 

2 

3 

4 

5 

j.  The  guide  properly  defines  each  Recon  Leader  Attribute 

1 

2 

3 

4 

5 

k.  The  guide  provides  enough  job  aids  to  meet  instructor  needs 

1 

2 

3 

4 

5 

1.  The  guide  is  tailored  to  the  ARC  instructional  environment 

1 

2 

3 

4 

5 

m.  The  guide  contains  the  right  level  of  detail  for  instructors 

1 

2 

3 

4 

5 

n.  The  guide  uses  language  and  terms  that  are  familiar  to  the  cadre 

1 

2 

3 

4 

5 

o.  The  guide  avoids  unnecessary  or  excess  infonnation 

1 

2 

3 

4 

5 

p.  The  guide  is  well  organized  and  easy  to  follow 

1 

2 

3 

4 

5 

q.  The  guide  is  easy  for  the  ARC  cadre  to  use 

1 

2 

3 

4 

5 

r.  The  guide  is  formatted  and  packaged  attractively 

1 

2 

3 

4 

5 

s.  The  guide  should  be  pocket  sized  for  easy  carrying 

1 

2 

3 

4 

5 

t.  The  guide  is  a  valuable  addition  to  the  ARC  toolkit 

1 

2 

3 

4 

5 

u.  The  guide  should  be  used  routinely  by  the  ARC  cadre 

1 

2 

3 

4 

5 

Comments  and  Suggestions: 

5.  What  could  be  added  to  the  guide  to  better  assist  you  or  the  instructors? 


6.  What  could  be  eliminated  from  the  guide  without  degrading  its  usefulness? 
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7.  How  do  you  feel  about  the  clarity  and  usefulness  of  the 
sections  of  the  guide’s  instructional  materials? 

Circle  One  for  Each  Item 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

a.  I  found  the  Utilization  Scenario  clear 

1 

2 

3 

4 

5 

b.  I  found  the  Utilization  Scenario  useful 

1 

2 

3 

4 

5 

c.  I  found  the  P2P  Training  Basics  section  clear 

1 

2 

3 

4 

5 

d.  I  found  the  P2P  Training  Basics  section  useful 

1 

2 

3 

4 

5 

e.  I  found  the  ARC  P2P  Training  Model  clear 

1 

2 

3 

4 

5 

f.  I  found  the  ARC  P2P  Training  Model  useful 

1 

2 

3 

4 

5 

g.  I  found  the  P2P  Training  Best  Practices  section  clear 

1 

2 

3 

4 

5 

h.  I  found  the  P2P  Training  Best  Practices  section  useful 

1 

2 

3 

4 

5 

i.  I  found  the  Setting  P2P  Training  Conditions  section  clear 

1 

2 

3 

4 

5 

j.  I  found  the  Setting  P2P  Training  Conditions  section  useful 

1 

2 

3 

4 

5 

k.  I  found  the  Recon  Leader  Skills  and  Attributes  section  clear 

1 

2 

3 

4 

5 

1.  I  found  the  Recon  Leader  Skills  and  Attributes  section  useful 

1 

2 

3 

4 

5 

m.  I  found  the  P2P  Training  Techniques  section  clear 

1 

2 

3 

4 

5 

n.  I  found  the  P2P  Training  Techniques  section  useful 

1 

2 

3 

4 

5 

o.  I  found  the  P2P  Training  Assessment  Best  Practices  section  clear 

1 

2 

3 

4 

5 

p.  I  found  the  P2P  Training  Assessment  Best  Practices  section  useful 

1 

2 

3 

4 

5 

q.  I  found  the  P2P  Training  Assessment  Techniques  section  clear 

1 

2 

3 

4 

5 

r.  I  found  the  P2P  Training  Assessment  Techniques  section  useful 

1 

2 

3 

4 

5 

Comments  and  Suggestions: 

8.  How  do  you  feel  about  the  clarity  and  usefulness  of  the 
job  aids  in  the  guide? 

Circle  One  for  Each  Item 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

a.  I  found  Recognizing  P2P  Training  Opportunities  clear 

1 

2 

3 

4 

5 

b.  I  found  Recognizing  P2P  Training  Opportunities  useful 

1 

2 

3 

4 

5 

c.  I  found  Using  Guiding  Questions  clear 

1 

2 

3 

4 

5 

d.  I  found  Using  Guiding  Questions  useful 

1 

2 

3 

4 

5 

e.  I  found  Exploiting  P2P  Training  Potential  clear 

1 

2 

3 

4 

5 

f.  I  found  Exploiting  P2P  Training  Potential  useful 

1 

2 

3 

4 

5 

g.  I  found  Using  Empowering  Statements  clear 

1 

2 

3 

4 

5 

h.  I  found  Using  Empowering  Statements  useful 

1 

2 

3 

4 

5 

i.  I  found  Deterring  Threats  to  Training  clear 

1 

2 

3 

4 

5 

j.  I  found  Deterring  Threats  to  Training  useful 

1 

2 

3 

4 

5 

k.  I  found  Assessment  of  Student  clear 

1 

2 

3 

4 

5 

1.  I  found  Assessment  of  Student  useful 

1 

2 

3 

4 

5 

m.  I  found  Assessment  of  Instructor  clear 

1 

2 

3 

4 

5 

n.  I  found  Assessment  of  Instructor  useful 

1 

2 

3 

4 

5 

o.  I  found  the  Hotwash/AAR  Job  Aid  clear 

1 

2 

3 

4 

5 

p.  I  found  the  Hotwash/AAR  Job  Aid  useful 

1 

2 

3 

4 

5 

Comments  and  Suggestions: 
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9.  How  do  vou  feel  about  the  suitability  of  the  assessment 
job  aids  found  in  the  quide? 

Circle  One  for  Each  Item 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

a.  The  Assessment  of  Student  job  aids  measure  the  right  skills 

1 

2 

3 

4 

5 

b.  The  Assessment  of  Student  job  aids  measure  the  right  attributes 

1 

2 

3 

4 

5 

c.  The  Assessment  of  Student  job  aids  take  too  long  to  implement 

1 

2 

3 

4 

5 

d.  The  Assessment  of  Student  job  aids  can  be  used  in  the  classroom 

1 

2 

3 

4 

5 

e.  The  Assessment  of  Student  job  aids  can  be  used  in  field  exercises 

1 

2 

3 

4 

5 

f.  The  Assessment  of  Student  job  aids  can  be  used  for  counseling 

1 

2 

3 

4 

5 

g.  The  Assessment  of  Student  job  aids  can  be  used  at  the  end  of  a  block 

1 

2 

3 

4 

5 

h.  The  Assessment  of  Student  job  aids  can  be  used  at  the  course’s  end 

1 

2 

3 

4 

5 

i.  The  Assessment  of  Student  job  aids  work  well  in  5x8  format 

1 

2 

3 

4 

5 

j.  The  Assessment  of  Instructor  job  aid  measures  the  right  attributes 

1 

2 

3 

4 

5 

k.  The  Assessment  of  Instructor  job  aid  takes  too  long  to  implement 

1 

2 

3 

4 

5 

1.  The  Assessment  of  Instructor  job  aid  can  be  used  at  end  of  a  block 

1 

2 

3 

4 

5 

m.  The  Assessment  of  Instructor  job  aid  can  be  used  at  the  course’s  end 

1 

2 

3 

4 

5 

n.  The  Assessment  of  Instructor  job  aid  works  well  in  5x8  format 

1 

2 

3 

4 

5 

Comments  and  Suggestions: 

10.  What  do  you  like  best  about  the  Instructor’s  P2P  Training  Guide? 


11.  What  do  you  like  least  about  the  Instructor’s  P2P  Training  Guide? 


12.  What  are  the  primary  benefits  of  using  the  guide? 


D-4 


13.  How  would  you  improve  the  following  parts  of  the  guide? 

Utilization  Scenario: 

P2P  Training  Basics: 

P2P  Training  Process  (diagram): 

P2P  Training  Guidelines/Techniques: 

Recon  Leader  Skills  and  Attributes: 

P2P  Training  Assessment  Guidelines/Techniques: 

Job  Aids  for  Training: 

Job  Aids  for  Assessment: 

Hotwash/AAR  Job  Aid: 


14.  What  other  comments  or  suggestions  do  you  have? 


Thank  you  for  your  feedback! 
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Appendix  E 


ARC-P2P  Training  Formative  Evaluation 

QUERY  GUIDE  -  END-OF-BLOCK  FEEDBACK  SESSION 


Instructions  for  Researcher(s): 

1 .  As  an  attendee  at  the  end-of-block  feedback  sessions,  you  can  expect  opportunities  to 
query  the  cadre  about  their  use  of  the  Instructor’s  P2P  Training  Guide.  Your  queries  will 
complement  the  primary  discussion  of  instructional  issues. 

2.  The  purpose  of  the  query  process  is  to  obtain  instructors’  opinions  and  suggestions  about 
the  Instructor’s  P2P  Training  Guide,  based  on  their  experiences  while  using  the  guide  in 
conjunction  with  their  ARC  training  activities. 

3.  Use  the  questions  that  start  on  the  next  page  as  a  guide,  not  a  rigid  protocol.  Stay  flexible 
so  you  can  cover  issues  and  concerns  of  special  interest  to  the  instructors.  You  may  not 
be  able  to  cover  all  the  questions  of  interest. 

4.  Feel  free  to  ask  other  questions  of  your  own,  especially  for  follow-up  or  clarification. 

5.  Take  along  several  copies  of  the  Instructor’s  P2P  Training  Guide  to  help  facilitate  the 
discussion,  as  appropriate. 

6.  Take  thorough  notes  during  the  session,  preferably  with  the  help  of  a  fellow  researcher. 

7.  You  may  audio-record  the  discussion  as  a  backup  to  the  written  notes.  If  you  do  so,  be 
sure  to  ask  the  group  if  anyone  objects  to  the  recording. 

8.  Within  3  days,  use  your  notes  to  compile  a  detailed  summary  in  a  Microsoft  Word  file. 
Organize  your  detailed  summary  by  questions. 

Rules  of  the  Road: 

1 .  When  possible,  coordinate  in  advance  with  the  session  leader  (e.g.,  Course  Leader)  to 
agree  on  workable  procedures. 

2.  If  discussion  of  the  guide  is  part  of  the  agenda,  try  to  concentrate  your  queries  in  the 
block  of  time  designated  for  the  research  dialogue. 

3.  If  there  is  no  dedicated  block  of  time  for  discussing  the  guide,  look  for  logical  openings 
to  relate  the  questions  in  this  guide  to  the  topics  under  discussion. 

4.  Respect  the  session  leader’s  role  as  the  person  in  charge,  unless  he  clearly  turns  the  floor 
over  to  the  research  team. 

5.  If  time  limits  have  been  announced,  watch  the  clock  so  the  session  ends  on  time. 

6.  Avoid  interrupting  the  primary  cadre  dialogue  unless  there  is  a  compelling  reason  for 
seizing  the  moment. 

7.  Be  alert  for  potential  sensitivities  of  the  instructors,  so  you  can  avoid  putting  them  on  the 
defense  in  public. 
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ADMINISTRATIVE  DATA 


Block  #: _  Date: _  #  Instructors: _ 

Researcher(s): _  Location: _ 

HANDOUT:  If  the  opportunity  arises,  hand  out  copies  of  the  Instructor’s  P2P  Training  Guide  to 
participants  who  would  like  one. 

QUESTIONS  OF  INTEREST 

1 )  How  well  did  the  guide  help  you  understand  P2P  training  methods? 

2)  Did  the  guide  help  you  focus  training  on  recon  leader  skills  and  attributes?  Please  explain. 

3)  How  did  the  guide  help  you  prepare  for  P2P  training  activities?  What  job  aids  did  you  use? 

4)  How  did  the  guide  help  you  improve  the  ARC  training  activities? 

5)  Did  the  guide  help  you  assess  the  students’  skills  and  attributes?  Please  explain. 

6)  How  did  the  guide  help  you  improve  assessment  of  students?  What  job  aids  did  you  use? 

7)  How  did  the  guide  help  you  provide  feedback  to  students?  What  job  aids  did  you  use? 

8)  Which  job  aid(s)  did  you  find  especially  useful?  How  did  you  use  them? 

9)  Did  the  guide  help  you  track  the  progress  of  individual  students?  Please  explain. 

10)  How  much  time  did  it  take  to  study  and  employ  the  materials  in  the  guide?  Too  much? 

1 1)  What  did  you  like  most  about  the  guide?  What  did  you  like  least? 

12)  How  do  you  think  ARC  instructors  can  best  use  the  guide? 

13)  What  are  the  benefits  of  using  the  guide? 

14)  Did  you  encounter  any  problems  while  using  the  guide?  Please  explain. 

15)  How  would  you  improve  the  guide  -  contents,  organization,  packaging? 
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